-- Logs begin at Wed 2022-08-24 08:35:12 UTC, end at Wed 2022-08-24 10:29:09 UTC. -- Aug 24 09:06:47.145584 dv-cinder-850321-5-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Aug 24 09:06:47.152778 dv-cinder-850321-5-iscsi devstack@keystone.service[81590]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Aug 24 09:06:47.154763 dv-cinder-850321-5-iscsi devstack@keystone.service[81590]: *** Starting uWSGI 2.0.18-debian (64bit) on [Wed Aug 24 09:06:47 2022] *** Aug 24 09:06:47.154763 dv-cinder-850321-5-iscsi devstack@keystone.service[81590]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 24 09:06:47.154763 dv-cinder-850321-5-iscsi devstack@keystone.service[81590]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 24 09:06:47.154763 dv-cinder-850321-5-iscsi devstack@keystone.service[81590]: nodename: dv-cinder-850321-5-iscsi Aug 24 09:06:47.154763 dv-cinder-850321-5-iscsi devstack@keystone.service[81590]: machine: x86_64 Aug 24 09:06:47.154763 dv-cinder-850321-5-iscsi devstack@keystone.service[81590]: clock source: unix Aug 24 09:06:47.154763 dv-cinder-850321-5-iscsi devstack@keystone.service[81590]: pcre jit disabled Aug 24 09:06:47.154763 dv-cinder-850321-5-iscsi devstack@keystone.service[81590]: detected number of CPU cores: 4 Aug 24 09:06:47.155288 dv-cinder-850321-5-iscsi devstack@keystone.service[81590]: current working directory: / Aug 24 09:06:47.155288 dv-cinder-850321-5-iscsi devstack@keystone.service[81590]: detected binary path: /usr/bin/uwsgi-core Aug 24 09:06:47.155288 dv-cinder-850321-5-iscsi devstack@keystone.service[81590]: your processes number limit is 47680 Aug 24 09:06:47.155288 dv-cinder-850321-5-iscsi devstack@keystone.service[81590]: your memory page size is 4096 bytes Aug 24 09:06:47.155288 dv-cinder-850321-5-iscsi devstack@keystone.service[81590]: detected max file descriptor number: 2048 Aug 24 09:06:47.155288 dv-cinder-850321-5-iscsi devstack@keystone.service[81590]: lock engine: pthread robust mutexes Aug 24 09:06:47.155288 dv-cinder-850321-5-iscsi devstack@keystone.service[81590]: thunder lock: enabled Aug 24 09:06:47.155288 dv-cinder-850321-5-iscsi devstack@keystone.service[81590]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Aug 24 09:06:47.155288 dv-cinder-850321-5-iscsi devstack@keystone.service[81590]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 24 09:06:47.195517 dv-cinder-850321-5-iscsi devstack@keystone.service[81590]: Python main interpreter initialized at 0x556a534589c0 Aug 24 09:06:47.195517 dv-cinder-850321-5-iscsi devstack@keystone.service[81590]: python threads support enabled Aug 24 09:06:47.195517 dv-cinder-850321-5-iscsi devstack@keystone.service[81590]: your server socket listen backlog is limited to 100 connections Aug 24 09:06:47.195517 dv-cinder-850321-5-iscsi devstack@keystone.service[81590]: your mercy for graceful operations on workers is 90 seconds Aug 24 09:06:47.195908 dv-cinder-850321-5-iscsi devstack@keystone.service[81590]: mapped 403077 bytes (393 KB) for 2 cores Aug 24 09:06:47.195985 dv-cinder-850321-5-iscsi devstack@keystone.service[81590]: *** Operational MODE: preforking *** Aug 24 09:06:47.196260 dv-cinder-850321-5-iscsi devstack@keystone.service[81590]: *** uWSGI is running in multiple interpreter mode *** Aug 24 09:06:47.196260 dv-cinder-850321-5-iscsi devstack@keystone.service[81590]: spawned uWSGI master process (pid: 81590) Aug 24 09:06:47.196469 dv-cinder-850321-5-iscsi systemd[1]: Started Devstack devstack@keystone.service. Aug 24 09:06:47.197526 dv-cinder-850321-5-iscsi devstack@keystone.service[81590]: spawned uWSGI worker 1 (pid: 81602, cores: 1) Aug 24 09:06:47.197526 dv-cinder-850321-5-iscsi devstack@keystone.service[81590]: spawned uWSGI worker 2 (pid: 81603, cores: 1) Aug 24 09:06:47.197526 dv-cinder-850321-5-iscsi devstack@keystone.service[81590]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 24 09:06:51.495519 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 24 09:06:51.495519 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: from cryptography.utils import int_from_bytes Aug 24 09:06:51.496385 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 24 09:06:51.496385 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: from cryptography.utils import int_from_bytes Aug 24 09:06:51.496385 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 24 09:06:51.496385 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: from cryptography.utils import int_from_bytes Aug 24 09:06:51.496913 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 24 09:06:51.496913 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: from cryptography.utils import int_from_bytes Aug 24 09:06:51.548903 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 24 09:06:51.549083 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 24 09:06:51.549283 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] command line args: [] {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 24 09:06:51.549558 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 24 09:06:51.549740 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] ================================================================================ {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 24 09:06:51.550001 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] admin_token = **** {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:06:51.550251 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] config_dir = [] {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:06:51.550335 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 24 09:06:51.550450 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:06:51.550540 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 24 09:06:51.550640 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] config_source = [] {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:06:51.550727 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] command line args: [] {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 24 09:06:51.550826 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] debug = True {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:06:51.550939 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 24 09:06:51.551030 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:06:51.551325 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ================================================================================ {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 24 09:06:51.551399 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] admin_token = **** {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:06:51.551458 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:06:51.551544 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:06:51.551642 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] config_dir = [] {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:06:51.551863 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:06:51.551948 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:06:51.552017 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] config_source = [] {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:06:51.552083 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:06:51.552148 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] debug = True {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:06:51.552321 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] list_limit = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:06:51.552398 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:06:51.552550 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] log_config_append = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:06:51.552633 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:06:51.552706 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:06:51.552778 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:06:51.552866 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] log_dir = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:06:51.552981 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:06:51.553056 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] log_file = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:06:51.553234 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:06:51.553309 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:06:51.553467 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] list_limit = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:06:51.553600 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:06:51.553682 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] log_config_append = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:06:51.553810 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:06:51.553936 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:06:51.554020 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] log_dir = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:06:51.554100 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:06:51.554177 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] log_file = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:06:51.554302 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:06:51.554302 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:06:51.554503 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:06:51.554591 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:06:51.554673 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:06:51.554797 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:06:51.554875 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:06:51.554998 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:06:51.555076 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:06:51.555238 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:06:51.555326 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:06:51.555445 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:06:51.555521 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:06:51.555692 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:06:51.555775 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:06:51.555944 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:06:51.556032 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:06:51.556180 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:06:51.556263 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:06:51.556415 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:06:51.556502 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:06:51.556649 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:06:51.556883 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:06:51.556968 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.16/identity {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:06:51.557114 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] publish_errors = False {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:06:51.557199 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:06:51.557354 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:06:51.557434 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:06:51.557592 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:06:51.557679 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:06:51.557776 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:06:51.558031 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:06:51.558118 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.16/identity {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:06:51.558274 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] publish_errors = False {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:06:51.558448 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:06:51.558529 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:06:51.558608 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] standard_threads = False {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:06:51.558729 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:06:51.558841 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:06:51.558931 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:06:51.559093 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:06:51.559181 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:06:51.559260 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:06:51.559368 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:06:51.559454 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] use_journal = False {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:06:51.559629 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] standard_threads = False {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:06:51.559714 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] use_json = False {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:06:51.559814 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] use_stderr = False {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:06:51.559892 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:06:51.559997 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] use_syslog = False {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:06:51.560103 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:06:51.560235 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:06:51.560322 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:06:51.560448 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] use_journal = False {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:06:51.560526 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.560626 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] use_json = False {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:06:51.560727 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.560817 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] use_stderr = False {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:06:51.560968 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] use_syslog = False {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:06:51.561048 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.561201 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:06:51.561280 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.561972 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.562136 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.562213 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.562289 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.562471 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.562558 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.562713 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.562800 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.562968 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.563049 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.563210 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.563297 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.563464 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.563613 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.563721 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.563819 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.563977 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.564100 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.564234 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.564324 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.564475 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.564555 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.564719 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.564803 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.564959 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.565037 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.565197 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.565280 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.565449 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.565526 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.565699 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.565778 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.565940 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.566015 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.566196 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.566273 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.566455 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.566534 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.566705 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.566782 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.566944 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.567020 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.567181 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.567258 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.567417 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.567494 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.567678 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.567771 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.567921 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.567996 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.568163 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.568241 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.568400 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.568476 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.568634 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.568713 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.568866 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.568942 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.569114 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.569191 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.569363 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.569441 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.569593 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.569668 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.569826 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.569906 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.570083 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.570172 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.570277 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.570376 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.570520 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.570609 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.570753 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.570834 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.571130 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.571217 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.571278 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.571350 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.571473 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.571557 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.571686 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.571764 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.571920 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.572013 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.572155 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.572241 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.572383 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.572505 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.572621 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.572723 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.572842 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.572928 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.573109 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.573249 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.573360 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.573447 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.573555 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.573640 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.573792 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.573879 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.574031 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.574109 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.574268 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.574360 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.574507 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.574588 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.574746 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.574855 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.574987 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.575058 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.575227 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.575317 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.575465 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.575552 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.575729 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.575813 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.575964 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.576045 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.576219 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.576305 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.576458 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.576550 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.576697 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.576779 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.576930 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.577017 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.577223 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.577818 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.578003 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.578140 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.578232 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.578303 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.578467 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.578565 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.578707 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.578896 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.578978 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.579049 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.579172 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.579261 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.579412 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.579491 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.579692 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.579818 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.586882 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.587046 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.587125 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.587205 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.587277 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.587348 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.587419 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.587497 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.587497 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.587651 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.587724 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.587794 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.587875 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.587961 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.588045 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.588114 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.588114 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.588219 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.588278 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.588337 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.588402 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.588462 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.588525 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.588585 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.588647 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.588707 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.588777 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.588842 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.588901 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.588959 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.589017 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.589075 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.589075 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.589190 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.589248 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.589306 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.589365 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.589423 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.589488 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.589548 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.589607 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.589666 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.589724 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.589782 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.589846 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.589906 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.589906 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.590023 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.590082 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.590139 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.590204 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.590264 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.590322 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.590379 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.590437 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.590500 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.590566 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.590631 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] auth.external = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.590691 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.590749 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.590807 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.590864 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.590930 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.590989 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.591047 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] auth.external = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.591134 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] auth.password = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.591201 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.591262 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] auth.token = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.591321 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.591380 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.591446 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.591506 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.591611 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] auth.password = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.591678 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.591738 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] auth.token = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.591797 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.591869 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.591931 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.591990 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.592221 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.592294 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.592364 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.592425 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.592484 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] credential.caching = True {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.592542 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.592600 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.592658 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.592724 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.592836 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.592934 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.593015 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] credential.caching = True {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.593094 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.593167 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.593246 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.593320 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.593391 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.593468 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.593541 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.593612 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.593695 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.593768 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.593839 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.593911 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.593911 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.594046 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.594125 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.594199 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.594270 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.594341 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.594412 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.594482 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.594564 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.594638 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.594709 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.594780 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.594851 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.594929 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] federation.caching = True {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.595074 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.595150 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.595150 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.595278 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.595278 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] federation.caching = True {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.595411 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.595411 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.595550 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.595657 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.595733 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.595808 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.595880 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.595880 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.596017 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.596088 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.596159 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.596229 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.596299 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.596378 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.596450 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.596601 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.596682 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.596755 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.596827 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] identity.caching = True {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.596908 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.596982 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.597054 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.597125 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.597241 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] identity.caching = True {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.597324 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.597491 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.597573 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.597742 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.597822 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.597974 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.598054 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.598213 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.598295 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.598465 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.598545 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.598702 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.598791 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.598939 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.599022 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.599174 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.599254 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.599408 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.599487 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.599671 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.599754 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.599908 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.599995 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.600143 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.600231 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.600393 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.600473 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.600642 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.600726 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.600890 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.600976 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.601143 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.601222 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.601382 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.601462 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.601620 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.601707 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.601862 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.601942 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.602095 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.602174 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.602328 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.602413 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.602568 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.602647 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.602801 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.602880 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.603041 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.603129 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.603286 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.603530 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.603668 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.603767 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.603847 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.603935 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.604010 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.604088 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.604255 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.604335 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.604506 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.604595 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.604749 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.604828 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.604987 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.605066 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.605222 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.605309 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.605456 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.605534 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.605691 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.605769 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.605921 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.606062 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.606163 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.606309 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.606389 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.606553 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.606632 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.606809 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.606890 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.607064 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.607144 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.607301 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.607380 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.607543 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.607677 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.607802 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.607893 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.608037 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.608129 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.608271 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.608359 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.608564 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.608647 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.608805 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.608932 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.609078 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.609278 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] ldap.user = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.609361 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.609513 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.user = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.609665 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.609745 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.609949 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.610127 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.610207 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.610365 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.610445 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.610618 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.610708 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.610872 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.610953 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.611125 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.611205 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.611365 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.611453 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.611614 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.611752 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.611860 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.611939 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.615476 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.615631 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.615718 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.615794 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.615869 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.615943 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.616032 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.616110 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.616184 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.616262 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.616335 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.616407 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.616490 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.616567 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.616641 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.616714 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.616787 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.616859 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.616941 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.617016 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.617090 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.617164 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.617244 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.617319 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.617401 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.617477 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.617551 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.617624 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.617698 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.617771 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.617857 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.617934 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.618008 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.618081 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.618155 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.618228 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.618310 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.618373 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.618446 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.618519 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.618592 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.618664 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.618810 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.618895 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.618971 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.619044 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.619117 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.619190 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.619271 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.619347 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.619420 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.619494 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.619567 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.619702 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] resource.caching = True {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.619790 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] resource.caching = True {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.619868 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.619976 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.620052 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.620125 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.620198 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.620279 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.620354 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.620428 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.620500 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.620574 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.620676 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.620764 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.620829 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.620902 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.620976 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.621049 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.621123 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.621123 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] role.caching = True {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.621266 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.621340 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] role.driver = sql {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.621340 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.621473 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] role.caching = True {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.621547 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.621628 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.621703 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] role.driver = sql {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.621835 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.622027 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.622116 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.622323 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.622470 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.622576 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.622833 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.622914 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.623087 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.623323 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.623412 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.623576 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.623826 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.623909 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.624623 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.624814 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.624910 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.625269 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.625351 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.625536 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.625724 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.625813 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.626035 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.626183 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.626291 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.626549 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.626629 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.626789 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.627005 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.627097 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.627291 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.627372 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.627536 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.627670 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.627810 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.627937 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.628199 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.628290 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.628447 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.628538 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.628707 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.628813 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.628961 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.629044 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.629221 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.629311 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.629480 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.629563 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.629720 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.629801 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.629956 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.630047 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.630199 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] token.caching = True {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.630282 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.630442 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.630524 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.630675 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.630764 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.630912 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.630993 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.631147 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.631229 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.631402 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.631492 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.631671 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.631798 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.631996 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.632088 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.632168 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.632261 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.632409 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.632502 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.632646 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.632751 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.632885 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] token.caching = True {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.632979 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.633124 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.633206 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.633376 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.633458 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.633616 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.633699 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.633875 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.633955 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.634124 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.634211 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.634366 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.634450 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.634611 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.634693 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] database.connection = **** {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.634859 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.634951 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.635116 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.635198 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.635356 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.635440 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.635614 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.635699 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.635859 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.635941 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.636101 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.636183 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.636343 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.636425 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.636574 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.636654 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.636821 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.636903 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.637061 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.637143 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.637310 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.637392 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] database.connection = **** {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.637552 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.637649 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.637805 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.637887 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.638044 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.638126 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.638329 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.638529 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.638613 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.638689 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.638763 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.638835 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.638977 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.639069 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.639146 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 24 09:06:51.639220 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.639447 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.640271 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.640525 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.640767 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.641020 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.641258 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.641504 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.641739 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.641983 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.642223 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:06:51.642397 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 24 09:06:51.669616 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81603) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:06:51.669840 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81603) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:06:51.670070 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81603) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:06:51.670723 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81602) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:06:51.671499 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81602) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:06:51.671761 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81602) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:06:51.679118 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81603) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:06:51.680255 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81602) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:06:51.726055 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81602) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:06:51.726290 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81602) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:06:51.726408 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81603) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:06:51.726647 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81603) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:06:51.787711 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81602) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:06:51.787927 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81603) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:06:51.837000 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81602) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:06:51.837189 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81602) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:06:51.837800 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81603) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:06:51.838041 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81603) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:06:52.001582 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81602) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:06:52.001971 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81602) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:06:52.002063 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81603) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:06:52.002134 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81602) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:06:52.002212 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81603) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:06:52.002284 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81602) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:06:52.002422 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81603) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:06:52.002505 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81602) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:06:52.002580 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81603) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:06:52.002655 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81602) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:06:52.002729 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81603) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:06:52.002898 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81603) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:06:52.025447 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81602) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:06:52.025631 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81602) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:06:52.025819 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81603) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:06:52.025905 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81602) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:06:52.026057 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81603) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:06:52.033671 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81602) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:06:52.033764 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81603) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:06:52.033842 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81602) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:06:52.033918 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81603) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:06:52.033982 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81602) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:06:52.034058 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81603) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:06:52.034135 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81602) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:06:52.034210 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81603) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:06:52.034210 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81603) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:06:52.044134 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81602) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:06:52.044509 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81603) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:06:52.057387 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81602) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:06:52.057623 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81602) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:06:52.057851 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81602) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:06:52.058079 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81602) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:06:52.058301 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81602) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:06:52.063072 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81602) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:06:52.063072 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81602) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:06:52.064208 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81603) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:06:52.064450 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81603) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:06:52.064674 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81603) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:06:52.064894 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81603) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:06:52.065109 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81603) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:06:52.065324 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81603) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:06:52.065537 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81603) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:06:52.078438 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:06:52.078708 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 24 09:06:52.078990 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:06:52.079131 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:06:52.079216 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 24 09:06:52.079345 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 24 09:06:52.079460 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:06:52.079719 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:06:52.079805 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:06:52.079925 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 24 09:06:52.080218 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:06:52.080308 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:06:52.080712 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:06:52.080812 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:06:52.081232 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:06:52.081498 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:06:52.092163 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:06:52.092163 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:06:52.092163 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:06:52.092374 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:06:52.092453 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:06:52.092515 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:06:52.092515 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:06:52.092515 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:06:52.098430 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81602) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 09:06:52.099185 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81603) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 09:06:52.101354 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81602) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 09:06:52.101648 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:06:52.101939 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:06:52.102197 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81603) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 09:06:52.102283 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:06:52.102492 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:06:52.102576 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:06:52.102756 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:06:52.102850 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:06:52.103033 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:06:52.103117 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:06:52.103314 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:06:52.103395 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:06:52.103592 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:06:52.103727 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:06:52.103870 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:06:52.104008 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:06:52.104147 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:06:52.104284 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:06:52.104532 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:06:52.104811 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:06:52.105120 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:06:52.127484 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:06:52.128257 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:06:52.128974 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81602) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 09:06:52.129264 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:06:52.129774 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81603) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 09:06:52.130065 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:06:52.133942 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81602) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 09:06:52.134232 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:06:52.134521 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:06:52.134799 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81603) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 09:06:52.135085 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:06:52.135371 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:06:52.140924 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81602) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 09:06:52.141218 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:06:52.141782 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81603) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 09:06:52.142074 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:06:52.145329 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81602) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 09:06:52.145624 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:06:52.145915 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:06:52.146207 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:06:52.146289 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81603) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 09:06:52.146489 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:06:52.146572 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:06:52.146772 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:06:52.146854 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:06:52.147050 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:06:52.147182 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:06:52.147325 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:06:52.147407 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:06:52.147686 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:06:52.147961 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:06:52.148232 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:06:52.165035 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:06:52.165319 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:06:52.165815 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:06:52.166100 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:06:52.169207 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81602) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 09:06:52.170011 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81603) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 09:06:52.172997 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:06:52.173301 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:06:52.173851 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:06:52.174152 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:06:52.177533 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81602) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 09:06:52.178370 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81603) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 09:06:52.180718 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81602) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 09:06:52.181577 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81603) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 09:06:52.184010 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:06:52.184316 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:06:52.187466 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:06:52.187558 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:06:52.187670 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:06:52.187747 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:06:52.187821 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:06:52.187893 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:06:52.187967 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:06:52.188050 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:06:52.188050 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:06:52.188050 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:06:52.201243 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81602) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 09:06:52.201535 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:06:52.201809 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:06:52.202109 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:06:52.202220 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81603) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 09:06:52.202439 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:06:52.202711 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:06:52.202975 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:06:52.209266 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:06:52.210181 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:06:52.210799 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:06:52.211082 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:06:52.211734 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:06:52.212017 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:06:52.213750 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81602) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 09:06:52.214038 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:06:52.214346 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:06:52.214645 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:06:52.214739 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81603) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 09:06:52.214931 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:06:52.215016 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:06:52.215265 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:06:52.215569 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:06:52.215869 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:06:52.226792 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81602) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 09:06:52.227082 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:06:52.227370 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:06:52.227680 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:06:52.227777 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81603) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 09:06:52.227967 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:06:52.228060 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:06:52.228249 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:06:52.228340 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:06:52.228533 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:06:52.228626 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:06:52.228816 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:06:52.229095 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:06:52.229373 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:06:52.245405 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81602) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 09:06:52.246020 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81603) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 09:06:52.248467 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81602) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 09:06:52.249106 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81603) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 09:06:52.251413 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:06:52.252049 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:06:52.252886 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:06:52.253547 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:06:52.254364 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81602) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 09:06:52.254652 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:06:52.254955 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:06:52.255060 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81603) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 09:06:52.255288 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:06:52.255604 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:06:52.261453 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:06:52.262080 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:06:52.262910 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81602) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 09:06:52.263489 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81603) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 09:06:52.265912 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:06:52.266207 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:06:52.266472 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:06:52.266577 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:06:52.266769 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:06:52.266862 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:06:52.267059 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:06:52.267350 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:06:52.274696 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81602) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 09:06:52.274993 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:06:52.275262 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81603) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 09:06:52.275355 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:06:52.275559 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:06:52.275900 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:06:52.282695 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81602) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 09:06:52.282986 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:06:52.283273 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:06:52.283364 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81603) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 09:06:52.283559 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:06:52.283665 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:06:52.283868 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:06:52.283961 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:06:52.284146 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:06:52.284228 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:06:52.287809 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:06:52.287988 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:06:52.288065 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:06:52.288129 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:06:52.288190 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:06:52.288266 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:06:52.288266 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:06:52.288380 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:06:52.288442 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:06:52.288502 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:06:52.288571 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:06:52.288635 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:06:52.288696 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:06:52.288756 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:06:52.288817 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:06:52.288878 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81602) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:06:52.288947 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:06:52.288947 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:06:52.320687 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x556a534589c0 pid: 81602 (default app) Aug 24 09:06:52.321288 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x556a534589c0 pid: 81603 (default app) Aug 24 09:06:52.324189 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7d64ef-faa9-4ff9-bcf2-d8b94101b14f None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:06:52.324480 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7d64ef-faa9-4ff9-bcf2-d8b94101b14f None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:06:52.324756 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7d64ef-faa9-4ff9-bcf2-d8b94101b14f None None] PATH_INFO: `/v3` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:06:52.328938 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1/1] 10.222.0.16 () {50 vars in 791 bytes} [Wed Aug 24 09:06:52 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:06:59.897672 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91e2f77-5c9e-4b33-aab9-cdf978e5c0ca None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:06:59.898213 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91e2f77-5c9e-4b33-aab9-cdf978e5c0ca None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:06:59.898213 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91e2f77-5c9e-4b33-aab9-cdf978e5c0ca None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:06:59.902427 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1/2] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:06:59 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:06:59.909336 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c568fe-1ce8-465c-98a7-721f6032e87f None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:06:59.909739 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c568fe-1ce8-465c-98a7-721f6032e87f None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:06:59.909885 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c568fe-1ce8-465c-98a7-721f6032e87f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:06:59.981685 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: INFO dbcounter [None req-f2c568fe-1ce8-465c-98a7-721f6032e87f None None] Registered counter for database keystone Aug 24 09:07:00.002354 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG oslo_db.sqlalchemy.engines [None req-f2c568fe-1ce8-465c-98a7-721f6032e87f None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81602) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 24 09:07:00.008693 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writer thread running {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 24 09:07:00.189018 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG passlib.handlers.bcrypt [None req-f2c568fe-1ce8-465c-98a7-721f6032e87f None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=81602) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 24 09:07:00.189360 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG passlib.handlers.bcrypt [None req-f2c568fe-1ce8-465c-98a7-721f6032e87f None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81602) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 24 09:07:00.244298 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-f2c568fe-1ce8-465c-98a7-721f6032e87f None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:07:00.245902 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-f2c568fe-1ce8-465c-98a7-721f6032e87f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:07:00.376712 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 2/3] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:06:59 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 468 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Aug 24 09:07:00.829333 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91e2f77-5c9e-4b33-aab9-cdf978e5c0ca None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:00.898598 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: INFO dbcounter [None req-d91e2f77-5c9e-4b33-aab9-cdf978e5c0ca None None] Registered counter for database keystone Aug 24 09:07:00.921210 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG oslo_db.sqlalchemy.engines [None req-d91e2f77-5c9e-4b33-aab9-cdf978e5c0ca None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81603) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 24 09:07:00.927331 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writer thread running {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 24 09:07:01.044895 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91e2f77-5c9e-4b33-aab9-cdf978e5c0ca None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:01.049735 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf02b24f-ad21-4336-85d4-30583a5acf37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:01.050402 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf02b24f-ad21-4336-85d4-30583a5acf37 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:01.050686 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf02b24f-ad21-4336-85d4-30583a5acf37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:01.050960 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf02b24f-ad21-4336-85d4-30583a5acf37 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:01.059020 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf02b24f-ad21-4336-85d4-30583a5acf37 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:01.110628 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: WARNING keystone.common.rbac_enforcer.enforcer [None req-bf02b24f-ad21-4336-85d4-30583a5acf37 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 24 09:07:01.110918 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf02b24f-ad21-4336-85d4-30583a5acf37 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:01.120326 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 2/4] 10.222.0.16 () {58 vars in 1324 bytes} [Wed Aug 24 09:07:00 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 24 09:07:01.126251 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c568fe-1ce8-465c-98a7-721f6032e87f None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:01.140625 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c568fe-1ce8-465c-98a7-721f6032e87f None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:01.145311 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec693ae-4aa9-474e-ace9-4ff23562bbe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:01.146422 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec693ae-4aa9-474e-ace9-4ff23562bbe8 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:01.146706 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec693ae-4aa9-474e-ace9-4ff23562bbe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:01.146979 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec693ae-4aa9-474e-ace9-4ff23562bbe8 admin admin] PATH_INFO: `/v3/projects/e359feabfca34b0e8852e124e6b4b5a1` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:01.148571 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec693ae-4aa9-474e-ace9-4ff23562bbe8 admin admin] RBAC: Authorizing `identity:get_project(project_id=e359feabfca34b0e8852e124e6b4b5a1)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:01.200439 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: WARNING keystone.common.rbac_enforcer.enforcer [None req-fec693ae-4aa9-474e-ace9-4ff23562bbe8 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 24 09:07:01.200782 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec693ae-4aa9-474e-ace9-4ff23562bbe8 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:01.205255 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 3/5] 10.222.0.16 () {56 vars in 1207 bytes} [Wed Aug 24 09:07:01 2022] GET /identity/v3/projects/e359feabfca34b0e8852e124e6b4b5a1 => generated 344 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:07:03.072390 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f020f31-8615-4779-a2eb-cf5a0b4bf549 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:03.072930 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f020f31-8615-4779-a2eb-cf5a0b4bf549 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:03.072930 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f020f31-8615-4779-a2eb-cf5a0b4bf549 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:03.077605 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 3/6] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:07:03 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 09:07:03.083192 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c3f841-2e88-4614-9494-702b2a94b158 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:03.083556 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c3f841-2e88-4614-9494-702b2a94b158 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:03.083789 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c3f841-2e88-4614-9494-702b2a94b158 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:03.103913 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-e5c3f841-2e88-4614-9494-702b2a94b158 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:07:03.105450 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-e5c3f841-2e88-4614-9494-702b2a94b158 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:07:03.113238 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 4/7] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:07:03 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 24 09:07:03.381725 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f020f31-8615-4779-a2eb-cf5a0b4bf549 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:03.394744 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f020f31-8615-4779-a2eb-cf5a0b4bf549 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:03.398946 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f870af0b-d68c-41e6-958e-737cd9cf5423 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:03.399550 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f870af0b-d68c-41e6-958e-737cd9cf5423 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:03.399883 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f870af0b-d68c-41e6-958e-737cd9cf5423 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:03.400163 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f870af0b-d68c-41e6-958e-737cd9cf5423 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:03.407955 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f870af0b-d68c-41e6-958e-737cd9cf5423 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:03.410081 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f870af0b-d68c-41e6-958e-737cd9cf5423 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:03.418953 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 4/8] 10.222.0.16 () {58 vars in 1324 bytes} [Wed Aug 24 09:07:03 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 09:07:03.425520 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c3f841-2e88-4614-9494-702b2a94b158 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:03.435042 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c3f841-2e88-4614-9494-702b2a94b158 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:03.439998 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d7cd0c-d4d6-462c-8f64-0d930fec5ab0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:03.440635 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d7cd0c-d4d6-462c-8f64-0d930fec5ab0 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:03.440916 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d7cd0c-d4d6-462c-8f64-0d930fec5ab0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:03.441191 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d7cd0c-d4d6-462c-8f64-0d930fec5ab0 admin admin] PATH_INFO: `/v3/users/d1828412cb04465dbfa15b26582aa6de` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:03.442511 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d7cd0c-d4d6-462c-8f64-0d930fec5ab0 admin admin] RBAC: Authorizing `identity:get_user(user_id=d1828412cb04465dbfa15b26582aa6de)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:03.444391 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d7cd0c-d4d6-462c-8f64-0d930fec5ab0 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:03.445943 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 5/9] 10.222.0.16 () {56 vars in 1198 bytes} [Wed Aug 24 09:07:03 2022] GET /identity/v3/users/d1828412cb04465dbfa15b26582aa6de => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:07:05.374259 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44fd3a5-3c63-4791-81fe-6a87d328eb4d None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:05.374817 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44fd3a5-3c63-4791-81fe-6a87d328eb4d None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:05.374817 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44fd3a5-3c63-4791-81fe-6a87d328eb4d None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:05.376179 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 5/10] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:07:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:07:05.381618 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513de775-f009-4dd5-955e-dce53f533480 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:05.382081 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513de775-f009-4dd5-955e-dce53f533480 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:05.382176 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513de775-f009-4dd5-955e-dce53f533480 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:05.399907 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-513de775-f009-4dd5-955e-dce53f533480 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:07:05.401093 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-513de775-f009-4dd5-955e-dce53f533480 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:07:05.436598 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4129dffc-7288-483a-bea4-4c9f0eff3bd0 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:05.437117 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4129dffc-7288-483a-bea4-4c9f0eff3bd0 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:05.437547 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4129dffc-7288-483a-bea4-4c9f0eff3bd0 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:05.438881 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 6/11] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:07:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 09:07:05.444160 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7ac4c5-6ff1-4bf4-a93d-45283b735f46 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:05.445751 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7ac4c5-6ff1-4bf4-a93d-45283b735f46 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:05.445751 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7ac4c5-6ff1-4bf4-a93d-45283b735f46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:05.481156 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 6/12] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:07:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:07:05.486650 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG passlib.handlers.bcrypt [None req-0b7ac4c5-6ff1-4bf4-a93d-45283b735f46 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=81603) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 24 09:07:05.487094 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG passlib.handlers.bcrypt [None req-0b7ac4c5-6ff1-4bf4-a93d-45283b735f46 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81603) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 24 09:07:05.532747 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-0b7ac4c5-6ff1-4bf4-a93d-45283b735f46 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:07:05.534328 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-0b7ac4c5-6ff1-4bf4-a93d-45283b735f46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:07:05.631571 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 7/13] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:07:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 09:07:05.747792 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513de775-f009-4dd5-955e-dce53f533480 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:05.771315 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:05.852402 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513de775-f009-4dd5-955e-dce53f533480 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:05.885568 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5198afa8-6941-4e3a-8608-2c9061af77dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:05.886149 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5198afa8-6941-4e3a-8608-2c9061af77dd None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:05.886435 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5198afa8-6941-4e3a-8608-2c9061af77dd None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:05.886715 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5198afa8-6941-4e3a-8608-2c9061af77dd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:05.896949 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7ac4c5-6ff1-4bf4-a93d-45283b735f46 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:05.898104 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5198afa8-6941-4e3a-8608-2c9061af77dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:07:05.907201 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-5198afa8-6941-4e3a-8608-2c9061af77dd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:07:05.907201 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:07:05.907201 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:07:05.907201 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:07:05.907201 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:07:05.907201 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:07:05.907201 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:07:05.907201 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:07:05.907201 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:07:05.907201 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:07:05.907201 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:07:05.907201 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:07:05.907201 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:07:05.907201 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:07:05.907201 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:07:05.907201 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:07:05.908392 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:07:05.908392 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:07:05.908392 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:07:05.908392 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:07:05.908392 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:07:05.908392 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:07:05.908392 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:07:05.908392 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:07:05.908392 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:07:05.908392 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:07:05.908392 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:07:05.908392 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 24 09:07:05.908392 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:07:05.908392 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:07:05.908392 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:07:05.908392 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:07:05.909244 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:07:05.909244 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:07:05.909244 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:07:05.909244 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:07:05.909244 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:07:05.909244 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:07:05.909244 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:07:05.909244 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:07:05.909244 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:07:05.909244 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:07:05.909244 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:07:05.909244 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 24 09:07:05.909244 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 7/14] 10.222.0.16 () {56 vars in 1096 bytes} [Wed Aug 24 09:07:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:07:05.913939 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5198afa8-6941-4e3a-8608-2c9061af77dd None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:05.994377 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:06.000717 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7ac4c5-6ff1-4bf4-a93d-45283b735f46 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:06.003064 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:06.020472 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5198afa8-6941-4e3a-8608-2c9061af77dd None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:06.041757 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056d4d12-f4a5-4b9e-9438-d63007a17d2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:06.042617 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056d4d12-f4a5-4b9e-9438-d63007a17d2d None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:06.042932 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056d4d12-f4a5-4b9e-9438-d63007a17d2d None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:06.043212 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056d4d12-f4a5-4b9e-9438-d63007a17d2d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:06.054498 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae40eab-9866-42a9-a15b-57a2ffc7d73d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:06.055148 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae40eab-9866-42a9-a15b-57a2ffc7d73d None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:06.055436 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae40eab-9866-42a9-a15b-57a2ffc7d73d None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:06.055744 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae40eab-9866-42a9-a15b-57a2ffc7d73d None admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:06.056422 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae40eab-9866-42a9-a15b-57a2ffc7d73d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:07:06.056728 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae40eab-9866-42a9-a15b-57a2ffc7d73d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:06.058122 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae40eab-9866-42a9-a15b-57a2ffc7d73d None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:06.069807 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 8/15] 10.222.0.16 () {56 vars in 1099 bytes} [Wed Aug 24 09:07:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:07:06.075484 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae40eab-9866-42a9-a15b-57a2ffc7d73d None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:06.112766 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056d4d12-f4a5-4b9e-9438-d63007a17d2d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:06.114300 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056d4d12-f4a5-4b9e-9438-d63007a17d2d None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:06.174820 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae40eab-9866-42a9-a15b-57a2ffc7d73d None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:06.208659 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d52bee3-e024-4d04-b43b-a75e3c76ef7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:06.209297 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d52bee3-e024-4d04-b43b-a75e3c76ef7d None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:06.209583 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d52bee3-e024-4d04-b43b-a75e3c76ef7d None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:06.209869 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d52bee3-e024-4d04-b43b-a75e3c76ef7d None admin] PATH_INFO: `/v3/users/d1828412cb04465dbfa15b26582aa6de` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:06.211017 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d52bee3-e024-4d04-b43b-a75e3c76ef7d None admin] RBAC: Authorizing `identity:get_user(user_id=d1828412cb04465dbfa15b26582aa6de)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:06.211380 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 8/16] 10.222.0.16 () {58 vars in 1282 bytes} [Wed Aug 24 09:07:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 09:07:06.212547 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d52bee3-e024-4d04-b43b-a75e3c76ef7d None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:06.218915 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 9/17] 10.222.0.16 () {56 vars in 1177 bytes} [Wed Aug 24 09:07:06 2022] GET /identity/v3/users/d1828412cb04465dbfa15b26582aa6de => generated 247 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:07:06.220139 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056d4d12-f4a5-4b9e-9438-d63007a17d2d None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:06.220252 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d52bee3-e024-4d04-b43b-a75e3c76ef7d None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:06.222904 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:06.235152 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:06.318807 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056d4d12-f4a5-4b9e-9438-d63007a17d2d None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:06.321206 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d52bee3-e024-4d04-b43b-a75e3c76ef7d None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:06.354842 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc448057-2968-48da-a569-0389e0e5818c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:06.355731 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc448057-2968-48da-a569-0389e0e5818c None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:06.356013 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc448057-2968-48da-a569-0389e0e5818c None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:06.356297 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc448057-2968-48da-a569-0389e0e5818c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:06.358777 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e332bc7a-f394-4080-9154-b3ea623e8cf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:06.359408 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e332bc7a-f394-4080-9154-b3ea623e8cf5 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:06.359721 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e332bc7a-f394-4080-9154-b3ea623e8cf5 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:06.359997 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e332bc7a-f394-4080-9154-b3ea623e8cf5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:06.361122 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e332bc7a-f394-4080-9154-b3ea623e8cf5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:06.362562 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e332bc7a-f394-4080-9154-b3ea623e8cf5 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:06.364135 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 10/18] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 24 09:07:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:07:06.369906 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e332bc7a-f394-4080-9154-b3ea623e8cf5 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:06.375663 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc448057-2968-48da-a569-0389e0e5818c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:06.377182 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc448057-2968-48da-a569-0389e0e5818c None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:06.395681 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-dc448057-2968-48da-a569-0389e0e5818c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:07:06.395681 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:07:06.395681 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:07:06.395681 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:07:06.395681 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:07:06.395681 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:07:06.395681 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:07:06.395681 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:07:06.395681 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:07:06.395681 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:07:06.395681 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:07:06.395681 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:07:06.395681 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:07:06.395681 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:07:06.395681 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:07:06.395681 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:07:06.396559 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:07:06.396559 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:07:06.396559 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:07:06.396559 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:07:06.396559 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:07:06.396559 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:07:06.396559 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:07:06.396559 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Aug 24 09:07:06.396559 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:07:06.396559 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:07:06.396559 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:07:06.396559 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:07:06.396559 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:07:06.396559 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:07:06.396559 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:07:06.396559 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:07:06.396559 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:07:06.397510 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:07:06.397510 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:07:06.397510 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Aug 24 09:07:06.397510 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 9/19] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 24 09:07:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:07:06.402417 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc448057-2968-48da-a569-0389e0e5818c None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:06.462292 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:06.465564 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:06.484183 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e332bc7a-f394-4080-9154-b3ea623e8cf5 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:06.506028 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc448057-2968-48da-a569-0389e0e5818c None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:06.518563 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af91b9ed-46f1-4b3b-924f-4eef95ef7253 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:06.519163 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af91b9ed-46f1-4b3b-924f-4eef95ef7253 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:06.519437 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af91b9ed-46f1-4b3b-924f-4eef95ef7253 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:06.519741 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af91b9ed-46f1-4b3b-924f-4eef95ef7253 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:06.520455 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af91b9ed-46f1-4b3b-924f-4eef95ef7253 None admin] RBAC: Adding query filter params (role.id=15f634d4a2d247ddab90c458e5e1c141, scope.domain.id=default, user.id=d1828412cb04465dbfa15b26582aa6de) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:07:06.520785 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af91b9ed-46f1-4b3b-924f-4eef95ef7253 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:06.522213 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af91b9ed-46f1-4b3b-924f-4eef95ef7253 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:06.529210 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 11/20] 10.222.0.16 () {56 vars in 1322 bytes} [Wed Aug 24 09:07:06 2022] GET /identity/v3/role_assignments?user.id=d1828412cb04465dbfa15b26582aa6de&scope.domain.id=default&role.id=15f634d4a2d247ddab90c458e5e1c141 => generated 233 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:07:06.547124 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568f6b27-0278-4fda-8c3a-83912d2a5280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:06.547793 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568f6b27-0278-4fda-8c3a-83912d2a5280 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:06.548078 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568f6b27-0278-4fda-8c3a-83912d2a5280 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:06.548353 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568f6b27-0278-4fda-8c3a-83912d2a5280 None admin] PATH_INFO: `/v3/domains` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:06.549017 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568f6b27-0278-4fda-8c3a-83912d2a5280 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:07:06.549325 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568f6b27-0278-4fda-8c3a-83912d2a5280 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:06.550772 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568f6b27-0278-4fda-8c3a-83912d2a5280 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:06.571892 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 10/21] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Aug 24 09:07:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:07:08.679799 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2b4ad4-3239-471a-84a8-6dd448c47972 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:08.680680 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2b4ad4-3239-471a-84a8-6dd448c47972 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:08.681140 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2b4ad4-3239-471a-84a8-6dd448c47972 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:08.682641 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 12/22] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:07:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:07:08.688085 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5a11b3-7fe2-4b0c-b0be-95b6aaf63ce5 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:08.688865 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5a11b3-7fe2-4b0c-b0be-95b6aaf63ce5 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:08.689330 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5a11b3-7fe2-4b0c-b0be-95b6aaf63ce5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:08.715631 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-6f5a11b3-7fe2-4b0c-b0be-95b6aaf63ce5 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:07:08.717685 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-6f5a11b3-7fe2-4b0c-b0be-95b6aaf63ce5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:07:08.805928 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 11/23] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:07:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:07:08.834981 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469f6954-f45c-411a-9d42-4bb6e1366f85 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:08.835561 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469f6954-f45c-411a-9d42-4bb6e1366f85 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:08.836056 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469f6954-f45c-411a-9d42-4bb6e1366f85 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:08.837622 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 13/24] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:07:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:07:08.843441 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a11bfa9-813e-4c91-a1be-c44e1b1848fb None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:08.843952 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a11bfa9-813e-4c91-a1be-c44e1b1848fb None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:08.844415 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a11bfa9-813e-4c91-a1be-c44e1b1848fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:08.872575 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-0a11bfa9-813e-4c91-a1be-c44e1b1848fb None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:07:08.874071 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-0a11bfa9-813e-4c91-a1be-c44e1b1848fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:07:08.925924 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:08.958032 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 12/25] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:07:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:07:09.139833 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d395f9e-1e15-41e8-be6d-68a75b6aadb0 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:09.144522 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d395f9e-1e15-41e8-be6d-68a75b6aadb0 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:09.144522 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d395f9e-1e15-41e8-be6d-68a75b6aadb0 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:09.144522 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 14/26] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:07:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:07:09.152912 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b586c8d5-92b6-49e2-9d56-b43131366803 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:09.153203 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b586c8d5-92b6-49e2-9d56-b43131366803 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:09.153484 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b586c8d5-92b6-49e2-9d56-b43131366803 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:09.180568 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-b586c8d5-92b6-49e2-9d56-b43131366803 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:07:09.181772 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-b586c8d5-92b6-49e2-9d56-b43131366803 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:07:09.259044 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 13/27] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:07:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:07:09.376752 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d395f9e-1e15-41e8-be6d-68a75b6aadb0 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:09.397961 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b586c8d5-92b6-49e2-9d56-b43131366803 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:09.553437 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d395f9e-1e15-41e8-be6d-68a75b6aadb0 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:09.563659 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b586c8d5-92b6-49e2-9d56-b43131366803 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:09.571552 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:09.601587 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1267ff-b8be-4788-b82a-f59981e36363 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:09.602526 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9012b28e-18d6-43b7-8a54-87ccc61b6cf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:09.602952 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1267ff-b8be-4788-b82a-f59981e36363 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:09.603162 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9012b28e-18d6-43b7-8a54-87ccc61b6cf6 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:09.603442 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9012b28e-18d6-43b7-8a54-87ccc61b6cf6 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:09.603748 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9012b28e-18d6-43b7-8a54-87ccc61b6cf6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:09.604074 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1267ff-b8be-4788-b82a-f59981e36363 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:09.604684 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1267ff-b8be-4788-b82a-f59981e36363 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:09.618530 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1267ff-b8be-4788-b82a-f59981e36363 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:09.620179 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1267ff-b8be-4788-b82a-f59981e36363 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:09.620776 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9012b28e-18d6-43b7-8a54-87ccc61b6cf6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:07:09.624500 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:09.631556 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-9012b28e-18d6-43b7-8a54-87ccc61b6cf6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:07:09.631556 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:07:09.631556 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:07:09.631556 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:07:09.631556 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:07:09.631556 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:07:09.631556 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:07:09.631556 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:07:09.631556 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:07:09.631556 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:07:09.631556 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:07:09.631556 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:07:09.631556 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:07:09.631556 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:07:09.631556 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:07:09.631556 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:07:09.632567 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:07:09.632567 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:07:09.632567 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:07:09.632567 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:07:09.632567 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:07:09.632567 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:07:09.632567 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:07:09.632567 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:07:09.632567 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:07:09.632567 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:07:09.632567 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:07:09.632567 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Aug 24 09:07:09.632567 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:07:09.632567 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:07:09.632567 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:07:09.632567 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:07:09.633326 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:07:09.633326 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:07:09.633326 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:07:09.633326 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:07:09.633326 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:07:09.633326 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:07:09.633326 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:07:09.633326 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:07:09.633326 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:07:09.633326 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:07:09.633326 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:07:09.633326 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Aug 24 09:07:09.636383 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 14/28] 10.222.0.16 () {56 vars in 1096 bytes} [Wed Aug 24 09:07:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 09:07:09.639289 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f34d78b-0782-40e8-bea8-4f9e5081ef3f None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:09.639289 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f34d78b-0782-40e8-bea8-4f9e5081ef3f None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:09.639289 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f34d78b-0782-40e8-bea8-4f9e5081ef3f None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:09.641110 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 15/29] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:07:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:07:09.644625 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-0d1267ff-b8be-4788-b82a-f59981e36363 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:07:09.644625 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:07:09.644625 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:07:09.644625 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:07:09.644625 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:07:09.644625 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:07:09.644625 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:07:09.644625 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:07:09.644625 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:07:09.644625 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:07:09.644625 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:07:09.644625 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:07:09.644625 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:07:09.644625 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:07:09.644625 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:07:09.644625 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:07:09.645446 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:07:09.645446 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:07:09.645446 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:07:09.645446 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:07:09.645446 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:07:09.645446 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:07:09.645446 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:07:09.645446 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 24 09:07:09.645446 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:07:09.645446 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:07:09.645446 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:07:09.645446 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:07:09.645446 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:07:09.645446 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:07:09.645446 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:07:09.645446 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:07:09.645446 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:07:09.646517 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:07:09.646517 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:07:09.646517 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 24 09:07:09.646517 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 15/30] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 24 09:07:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:07:09.648924 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1267ff-b8be-4788-b82a-f59981e36363 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:09.649098 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f34d78b-0782-40e8-bea8-4f9e5081ef3f None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:09.755847 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1267ff-b8be-4788-b82a-f59981e36363 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:09.788823 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f34d78b-0782-40e8-bea8-4f9e5081ef3f None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:09.790849 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49cb522-9e14-464a-92f9-1e75a0c57327 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:09.791639 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49cb522-9e14-464a-92f9-1e75a0c57327 None admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:09.792043 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49cb522-9e14-464a-92f9-1e75a0c57327 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:09.793097 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49cb522-9e14-464a-92f9-1e75a0c57327 None admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:09.793213 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49cb522-9e14-464a-92f9-1e75a0c57327 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:09.794750 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49cb522-9e14-464a-92f9-1e75a0c57327 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:09.827526 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a65b63e-093c-4037-a050-1144bd8f22c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:09.828082 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a65b63e-093c-4037-a050-1144bd8f22c1 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:09.828364 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a65b63e-093c-4037-a050-1144bd8f22c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:09.828636 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a65b63e-093c-4037-a050-1144bd8f22c1 None admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:09.829307 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a65b63e-093c-4037-a050-1144bd8f22c1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:07:09.829621 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a65b63e-093c-4037-a050-1144bd8f22c1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:09.831274 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a65b63e-093c-4037-a050-1144bd8f22c1 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:09.845046 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 16/31] 10.222.0.16 () {56 vars in 1099 bytes} [Wed Aug 24 09:07:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:07:09.845947 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a65b63e-093c-4037-a050-1144bd8f22c1 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:09.876194 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:09.952224 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a65b63e-093c-4037-a050-1144bd8f22c1 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:09.988838 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8177b470-3946-40e4-8512-658632c325dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:09.989503 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8177b470-3946-40e4-8512-658632c325dd None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:09.989783 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8177b470-3946-40e4-8512-658632c325dd None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:09.990055 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8177b470-3946-40e4-8512-658632c325dd None admin] PATH_INFO: `/v3/domains` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:09.990750 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8177b470-3946-40e4-8512-658632c325dd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:07:09.991058 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8177b470-3946-40e4-8512-658632c325dd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:09.992509 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8177b470-3946-40e4-8512-658632c325dd None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:10.009759 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 17/32] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Aug 24 09:07:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:07:10.009759 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565a33c3-bad7-429f-9ca5-60aad20abe4c None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:10.009759 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565a33c3-bad7-429f-9ca5-60aad20abe4c None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:10.009759 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565a33c3-bad7-429f-9ca5-60aad20abe4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:10.025668 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-565a33c3-bad7-429f-9ca5-60aad20abe4c None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:07:10.026965 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-565a33c3-bad7-429f-9ca5-60aad20abe4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:07:10.047353 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 16/33] 10.222.0.16 () {60 vars in 1131 bytes} [Wed Aug 24 09:07:09 2022] POST /identity/v3/roles => generated 219 bytes in 404 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:07:10.048787 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49cb522-9e14-464a-92f9-1e75a0c57327 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:10.145083 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 18/34] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:07:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:07:10.146829 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565a33c3-bad7-429f-9ca5-60aad20abe4c None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:10.171179 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:10.221102 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:10.273752 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49cb522-9e14-464a-92f9-1e75a0c57327 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:10.282121 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565a33c3-bad7-429f-9ca5-60aad20abe4c None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:10.328927 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc3e4b3-53a4-4831-a745-531332fd5602 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:10.329559 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc3e4b3-53a4-4831-a745-531332fd5602 None admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:10.329841 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc3e4b3-53a4-4831-a745-531332fd5602 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:10.330168 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc3e4b3-53a4-4831-a745-531332fd5602 None admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:10.332019 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc3e4b3-53a4-4831-a745-531332fd5602 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:10.333496 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc3e4b3-53a4-4831-a745-531332fd5602 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:10.358581 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad74b3c0-8543-4536-a8e3-e849cee26e72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:10.361366 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad74b3c0-8543-4536-a8e3-e849cee26e72 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:10.361745 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad74b3c0-8543-4536-a8e3-e849cee26e72 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:10.362109 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad74b3c0-8543-4536-a8e3-e849cee26e72 None admin] PATH_INFO: `/v3/users/d1828412cb04465dbfa15b26582aa6de` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:10.365340 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad74b3c0-8543-4536-a8e3-e849cee26e72 None admin] RBAC: Authorizing `identity:get_user(user_id=d1828412cb04465dbfa15b26582aa6de)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:10.366971 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad74b3c0-8543-4536-a8e3-e849cee26e72 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:10.368810 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 17/35] 10.222.0.16 () {56 vars in 1177 bytes} [Wed Aug 24 09:07:10 2022] GET /identity/v3/users/d1828412cb04465dbfa15b26582aa6de => generated 247 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:07:10.371641 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2ad5a6-f3bd-4b79-9e22-35c620ec7331 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:10.371641 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2ad5a6-f3bd-4b79-9e22-35c620ec7331 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:10.371641 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2ad5a6-f3bd-4b79-9e22-35c620ec7331 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:10.373217 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 18/36] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:07:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:07:10.374775 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2ad5a6-f3bd-4b79-9e22-35c620ec7331 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:10.496320 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2ad5a6-f3bd-4b79-9e22-35c620ec7331 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:10.522130 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:10.533478 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e08f6c9-de62-4d1a-a347-c05edf865681 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:10.534123 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e08f6c9-de62-4d1a-a347-c05edf865681 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:10.534408 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e08f6c9-de62-4d1a-a347-c05edf865681 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:10.534738 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e08f6c9-de62-4d1a-a347-c05edf865681 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:10.535989 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e08f6c9-de62-4d1a-a347-c05edf865681 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:10.537494 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e08f6c9-de62-4d1a-a347-c05edf865681 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:10.543959 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 19/37] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 24 09:07:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:07:10.543959 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277a8887-6ca6-4dd1-aa66-b6b0183db264 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:10.543959 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277a8887-6ca6-4dd1-aa66-b6b0183db264 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:10.543959 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277a8887-6ca6-4dd1-aa66-b6b0183db264 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:10.560264 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-277a8887-6ca6-4dd1-aa66-b6b0183db264 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:07:10.561817 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-277a8887-6ca6-4dd1-aa66-b6b0183db264 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:07:10.578283 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 19/38] 10.222.0.16 () {60 vars in 1141 bytes} [Wed Aug 24 09:07:10 2022] POST /identity/v3/projects => generated 301 bytes in 432 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:07:10.579251 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc3e4b3-53a4-4831-a745-531332fd5602 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:10.637067 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 20/39] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:07:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:07:10.644894 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277a8887-6ca6-4dd1-aa66-b6b0183db264 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:10.660075 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:10.705798 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc3e4b3-53a4-4831-a745-531332fd5602 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:10.753121 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdd4dcf-f243-41d4-a87e-1941ff35273e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:10.754106 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdd4dcf-f243-41d4-a87e-1941ff35273e None admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:10.754600 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdd4dcf-f243-41d4-a87e-1941ff35273e None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:10.755065 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdd4dcf-f243-41d4-a87e-1941ff35273e None admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:10.755947 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdd4dcf-f243-41d4-a87e-1941ff35273e None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:10.757884 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdd4dcf-f243-41d4-a87e-1941ff35273e None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:10.780747 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277a8887-6ca6-4dd1-aa66-b6b0183db264 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:10.804384 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:10.816244 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a8d301-3486-4e8f-b28d-08989cf64ff5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:10.816923 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a8d301-3486-4e8f-b28d-08989cf64ff5 None admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:10.817227 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a8d301-3486-4e8f-b28d-08989cf64ff5 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:10.817509 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a8d301-3486-4e8f-b28d-08989cf64ff5 None admin] PATH_INFO: `/v3/domains/default/users/d1828412cb04465dbfa15b26582aa6de/roles/15f634d4a2d247ddab90c458e5e1c141` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:10.819456 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a8d301-3486-4e8f-b28d-08989cf64ff5 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=d1828412cb04465dbfa15b26582aa6de, role_id=15f634d4a2d247ddab90c458e5e1c141)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:10.820952 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a8d301-3486-4e8f-b28d-08989cf64ff5 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:10.827629 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 20/40] 10.222.0.16 () {60 vars in 1131 bytes} [Wed Aug 24 09:07:10 2022] POST /identity/v3/roles => generated 225 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:07:10.829266 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501b9b53-2b2b-4612-90f3-c36b58f4f6e1 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:10.829266 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501b9b53-2b2b-4612-90f3-c36b58f4f6e1 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:10.829400 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501b9b53-2b2b-4612-90f3-c36b58f4f6e1 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:10.830967 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 21/41] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:07:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:07:10.837086 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572a3646-f69f-410c-add3-f7d1e0a1b939 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:10.837388 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572a3646-f69f-410c-add3-f7d1e0a1b939 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:10.837723 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572a3646-f69f-410c-add3-f7d1e0a1b939 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:10.857643 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-572a3646-f69f-410c-add3-f7d1e0a1b939 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:07:10.859282 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-572a3646-f69f-410c-add3-f7d1e0a1b939 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:07:10.905908 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 21/42] 10.222.0.16 () {58 vars in 1361 bytes} [Wed Aug 24 09:07:10 2022] PUT /identity/v3/domains/default/users/d1828412cb04465dbfa15b26582aa6de/roles/15f634d4a2d247ddab90c458e5e1c141 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:07:10.906764 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a8d301-3486-4e8f-b28d-08989cf64ff5 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:10.947882 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 22/43] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:07:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:07:11.034828 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a8d301-3486-4e8f-b28d-08989cf64ff5 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:11.071601 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0b47c4-58fa-4b4f-810a-d0de6d09d1c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:11.072279 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0b47c4-58fa-4b4f-810a-d0de6d09d1c8 None admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:11.072564 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0b47c4-58fa-4b4f-810a-d0de6d09d1c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:11.072845 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0b47c4-58fa-4b4f-810a-d0de6d09d1c8 None admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:11.073570 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0b47c4-58fa-4b4f-810a-d0de6d09d1c8 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:11.074966 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0b47c4-58fa-4b4f-810a-d0de6d09d1c8 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:11.142815 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 22/44] 10.222.0.16 () {60 vars in 1131 bytes} [Wed Aug 24 09:07:10 2022] POST /identity/v3/roles => generated 223 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:07:11.229486 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572a3646-f69f-410c-add3-f7d1e0a1b939 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:11.296184 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:11.342981 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572a3646-f69f-410c-add3-f7d1e0a1b939 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:11.378736 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3aa736-ccd9-4cab-bc05-b0fc30f11f95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:11.379378 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3aa736-ccd9-4cab-bc05-b0fc30f11f95 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:11.379765 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3aa736-ccd9-4cab-bc05-b0fc30f11f95 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:11.380041 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3aa736-ccd9-4cab-bc05-b0fc30f11f95 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:11.381163 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3aa736-ccd9-4cab-bc05-b0fc30f11f95 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:11.382619 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3aa736-ccd9-4cab-bc05-b0fc30f11f95 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:11.384486 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 23/45] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 24 09:07:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:07:11.390093 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0b47c4-58fa-4b4f-810a-d0de6d09d1c8 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:11.432689 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:11.489424 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0b47c4-58fa-4b4f-810a-d0de6d09d1c8 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:11.521760 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a3bad5-d626-4a57-bf33-1afe9877aee0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:11.522366 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a3bad5-d626-4a57-bf33-1afe9877aee0 None admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:11.522647 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a3bad5-d626-4a57-bf33-1afe9877aee0 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:11.522920 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a3bad5-d626-4a57-bf33-1afe9877aee0 None admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:11.523566 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a3bad5-d626-4a57-bf33-1afe9877aee0 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:11.525082 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a3bad5-d626-4a57-bf33-1afe9877aee0 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:11.604766 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 23/46] 10.222.0.16 () {60 vars in 1141 bytes} [Wed Aug 24 09:07:11 2022] POST /identity/v3/projects => generated 312 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:07:12.843283 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8fe472-56db-4a31-bae0-cb4e98b5f812 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:12.843861 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8fe472-56db-4a31-bae0-cb4e98b5f812 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:12.843957 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8fe472-56db-4a31-bae0-cb4e98b5f812 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:12.845249 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 24/47] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:07:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:07:12.850576 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8441f00-a3af-4643-9bb1-d6961e691a5c None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:12.850997 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8441f00-a3af-4643-9bb1-d6961e691a5c None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:12.851257 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8441f00-a3af-4643-9bb1-d6961e691a5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:12.973709 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-f8441f00-a3af-4643-9bb1-d6961e691a5c None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:07:12.974921 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-f8441f00-a3af-4643-9bb1-d6961e691a5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:07:13.041467 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 24/48] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:07:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:07:13.310065 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8fe472-56db-4a31-bae0-cb4e98b5f812 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:13.411828 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8fe472-56db-4a31-bae0-cb4e98b5f812 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:13.437723 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:13.445176 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15155e9-0b75-41ec-ba3b-a1aee29c9b69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:13.445815 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15155e9-0b75-41ec-ba3b-a1aee29c9b69 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:13.446090 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15155e9-0b75-41ec-ba3b-a1aee29c9b69 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:13.449312 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15155e9-0b75-41ec-ba3b-a1aee29c9b69 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:13.456926 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15155e9-0b75-41ec-ba3b-a1aee29c9b69 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:07:13.462003 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-a15155e9-0b75-41ec-ba3b-a1aee29c9b69 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:07:13.462003 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:07:13.462003 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:07:13.462003 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:07:13.462003 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:07:13.462003 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:07:13.462003 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:07:13.462003 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:07:13.462003 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:07:13.462003 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:07:13.462003 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:07:13.462003 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:07:13.462003 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:07:13.462003 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:07:13.462003 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:07:13.462003 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:07:13.462849 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:07:13.462849 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:07:13.462849 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:07:13.462849 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:07:13.462849 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:07:13.462849 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:07:13.462849 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:07:13.462849 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:07:13.462849 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:07:13.462849 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:07:13.462849 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:07:13.462849 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Aug 24 09:07:13.462849 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:07:13.462849 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:07:13.462849 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:07:13.462849 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:07:13.463666 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:07:13.463666 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:07:13.463666 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:07:13.463666 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:07:13.463666 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:07:13.463666 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:07:13.463666 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:07:13.463666 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:07:13.463666 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:07:13.463666 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:07:13.463666 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:07:13.463666 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Aug 24 09:07:13.463666 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 25/49] 10.222.0.16 () {56 vars in 1096 bytes} [Wed Aug 24 09:07:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:07:13.469394 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8441f00-a3af-4643-9bb1-d6961e691a5c None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:13.493249 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:13.521624 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b65ab39-311a-4da8-902d-bd26c516f3ee None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:13.521906 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b65ab39-311a-4da8-902d-bd26c516f3ee None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:13.522182 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b65ab39-311a-4da8-902d-bd26c516f3ee None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:13.523523 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 26/50] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:07:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 09:07:13.528848 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac1e61d-12c3-494f-8899-bbaccbd30c0b None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:13.529742 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac1e61d-12c3-494f-8899-bbaccbd30c0b None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:13.530021 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac1e61d-12c3-494f-8899-bbaccbd30c0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:13.547658 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-5ac1e61d-12c3-494f-8899-bbaccbd30c0b None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:07:13.548864 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-5ac1e61d-12c3-494f-8899-bbaccbd30c0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:07:13.567847 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8441f00-a3af-4643-9bb1-d6961e691a5c None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:13.601064 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2c38f8-adc7-4b05-ac7d-25e27326b2c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:13.601676 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2c38f8-adc7-4b05-ac7d-25e27326b2c0 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:13.601962 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2c38f8-adc7-4b05-ac7d-25e27326b2c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:13.602236 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2c38f8-adc7-4b05-ac7d-25e27326b2c0 None admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:13.602908 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2c38f8-adc7-4b05-ac7d-25e27326b2c0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:07:13.603214 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2c38f8-adc7-4b05-ac7d-25e27326b2c0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:13.604622 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2c38f8-adc7-4b05-ac7d-25e27326b2c0 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:13.613104 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 25/51] 10.222.0.16 () {56 vars in 1099 bytes} [Wed Aug 24 09:07:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:07:13.615112 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 27/52] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:07:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 09:07:13.618784 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2c38f8-adc7-4b05-ac7d-25e27326b2c0 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:13.698553 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:13.713638 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2c38f8-adc7-4b05-ac7d-25e27326b2c0 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:13.746980 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b10d25f-510d-4300-ac28-e50084cd5c33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:13.747603 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b10d25f-510d-4300-ac28-e50084cd5c33 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:13.748444 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b10d25f-510d-4300-ac28-e50084cd5c33 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:13.748728 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b10d25f-510d-4300-ac28-e50084cd5c33 None admin] PATH_INFO: `/v3/users/d1828412cb04465dbfa15b26582aa6de` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:13.749858 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b10d25f-510d-4300-ac28-e50084cd5c33 None admin] RBAC: Authorizing `identity:get_user(user_id=d1828412cb04465dbfa15b26582aa6de)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:13.751279 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b10d25f-510d-4300-ac28-e50084cd5c33 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:13.752858 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 26/53] 10.222.0.16 () {56 vars in 1177 bytes} [Wed Aug 24 09:07:13 2022] GET /identity/v3/users/d1828412cb04465dbfa15b26582aa6de => generated 247 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:07:13.758454 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac1e61d-12c3-494f-8899-bbaccbd30c0b None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:13.853852 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac1e61d-12c3-494f-8899-bbaccbd30c0b None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:13.871975 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:13.885398 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b10d25f-510d-4300-ac28-e50084cd5c33 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:13.888298 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fafeae-ff58-4e12-8fdd-7c83f006f88b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:13.888937 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fafeae-ff58-4e12-8fdd-7c83f006f88b None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:13.889217 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fafeae-ff58-4e12-8fdd-7c83f006f88b None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:13.889491 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fafeae-ff58-4e12-8fdd-7c83f006f88b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:13.890643 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fafeae-ff58-4e12-8fdd-7c83f006f88b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:13.892131 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fafeae-ff58-4e12-8fdd-7c83f006f88b None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:13.895486 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 28/54] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 24 09:07:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:07:13.899344 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fafeae-ff58-4e12-8fdd-7c83f006f88b None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:13.984973 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b10d25f-510d-4300-ac28-e50084cd5c33 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:13.998456 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fafeae-ff58-4e12-8fdd-7c83f006f88b None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:14.018731 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be19045-ce06-4271-a848-7418359f8806 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:14.019493 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be19045-ce06-4271-a848-7418359f8806 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:14.019867 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be19045-ce06-4271-a848-7418359f8806 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:14.020147 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be19045-ce06-4271-a848-7418359f8806 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:14.021316 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be19045-ce06-4271-a848-7418359f8806 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:14.022810 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be19045-ce06-4271-a848-7418359f8806 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:14.024472 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 27/55] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 24 09:07:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:07:14.030160 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be19045-ce06-4271-a848-7418359f8806 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:14.033340 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc7987f-e667-4f6b-9b6c-25d8d34940cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:14.033951 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc7987f-e667-4f6b-9b6c-25d8d34940cc None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:14.034232 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc7987f-e667-4f6b-9b6c-25d8d34940cc None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:14.034512 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc7987f-e667-4f6b-9b6c-25d8d34940cc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:14.035247 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc7987f-e667-4f6b-9b6c-25d8d34940cc None admin] RBAC: Adding query filter params (role.id=15f634d4a2d247ddab90c458e5e1c141, scope.domain.id=default, user.id=d1828412cb04465dbfa15b26582aa6de) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:07:14.035556 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc7987f-e667-4f6b-9b6c-25d8d34940cc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:14.037009 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc7987f-e667-4f6b-9b6c-25d8d34940cc None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:14.046301 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 29/56] 10.222.0.16 () {56 vars in 1322 bytes} [Wed Aug 24 09:07:13 2022] GET /identity/v3/role_assignments?user.id=d1828412cb04465dbfa15b26582aa6de&scope.domain.id=default&role.id=15f634d4a2d247ddab90c458e5e1c141 => generated 530 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:07:14.053858 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:14.135142 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be19045-ce06-4271-a848-7418359f8806 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:14.171093 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e71ffc5-cc21-4aea-8302-f9dfc86a5953 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:14.171749 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e71ffc5-cc21-4aea-8302-f9dfc86a5953 None admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:14.172031 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e71ffc5-cc21-4aea-8302-f9dfc86a5953 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:14.172303 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e71ffc5-cc21-4aea-8302-f9dfc86a5953 None admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:14.172965 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e71ffc5-cc21-4aea-8302-f9dfc86a5953 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:14.174431 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e71ffc5-cc21-4aea-8302-f9dfc86a5953 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:14.262297 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 28/57] 10.222.0.16 () {60 vars in 1140 bytes} [Wed Aug 24 09:07:14 2022] POST /identity/v3/projects => generated 298 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 09:07:16.160268 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6446f3b-7dcf-474d-a415-8836b8dfdca3 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:16.161094 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6446f3b-7dcf-474d-a415-8836b8dfdca3 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:16.161241 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6446f3b-7dcf-474d-a415-8836b8dfdca3 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:16.162626 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 30/58] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:07:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:07:16.167928 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c9c4f9-9e56-42da-9b15-311b428b29fd None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:16.168374 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c9c4f9-9e56-42da-9b15-311b428b29fd None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:16.168548 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c9c4f9-9e56-42da-9b15-311b428b29fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:16.187397 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-f4c9c4f9-9e56-42da-9b15-311b428b29fd None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:07:16.188624 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-f4c9c4f9-9e56-42da-9b15-311b428b29fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:07:16.252520 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 29/59] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:07:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:07:16.518761 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6446f3b-7dcf-474d-a415-8836b8dfdca3 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:16.568374 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:16.618951 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6446f3b-7dcf-474d-a415-8836b8dfdca3 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:16.651409 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0c39f2-cb1b-4b7e-900f-658f18219a5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:16.652058 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0c39f2-cb1b-4b7e-900f-658f18219a5f None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:16.652345 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0c39f2-cb1b-4b7e-900f-658f18219a5f None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:16.652622 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0c39f2-cb1b-4b7e-900f-658f18219a5f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:16.653725 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0c39f2-cb1b-4b7e-900f-658f18219a5f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:16.655755 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0c39f2-cb1b-4b7e-900f-658f18219a5f None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:16.657288 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 31/60] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 24 09:07:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:07:16.662565 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c9c4f9-9e56-42da-9b15-311b428b29fd None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:16.665626 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:16.757828 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c9c4f9-9e56-42da-9b15-311b428b29fd None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:16.790731 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c1906c-a7d2-4c76-8f05-7ef8b254b2ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:16.791329 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c1906c-a7d2-4c76-8f05-7ef8b254b2ea None admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:16.791635 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c1906c-a7d2-4c76-8f05-7ef8b254b2ea None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:16.791915 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c1906c-a7d2-4c76-8f05-7ef8b254b2ea None admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:16.792562 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c1906c-a7d2-4c76-8f05-7ef8b254b2ea None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:16.793999 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c1906c-a7d2-4c76-8f05-7ef8b254b2ea None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:16.893114 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 30/61] 10.222.0.16 () {60 vars in 1132 bytes} [Wed Aug 24 09:07:16 2022] POST /identity/v3/users => generated 275 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:07:19.837127 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0542df21-69ac-4d09-8ee6-c26f91477a36 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:19.838138 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0542df21-69ac-4d09-8ee6-c26f91477a36 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:19.838617 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0542df21-69ac-4d09-8ee6-c26f91477a36 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:19.840181 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 32/62] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:07:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:07:19.854032 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccf6317-1fa6-4d91-ae5d-b52b9d22f3a4 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:19.854984 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccf6317-1fa6-4d91-ae5d-b52b9d22f3a4 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:19.855469 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccf6317-1fa6-4d91-ae5d-b52b9d22f3a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:19.875810 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-9ccf6317-1fa6-4d91-ae5d-b52b9d22f3a4 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:07:19.877317 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-9ccf6317-1fa6-4d91-ae5d-b52b9d22f3a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:07:19.912910 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba2b899-a864-4efc-9639-139a7f90b9ea None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:19.913528 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba2b899-a864-4efc-9639-139a7f90b9ea None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:19.914016 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba2b899-a864-4efc-9639-139a7f90b9ea None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:19.915618 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 33/63] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:07:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:07:19.921855 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e928d4-e1d5-4a9d-aef2-55b17d06f5db None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:19.922611 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e928d4-e1d5-4a9d-aef2-55b17d06f5db None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:19.923108 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e928d4-e1d5-4a9d-aef2-55b17d06f5db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:19.952876 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-49e928d4-e1d5-4a9d-aef2-55b17d06f5db None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:07:19.954077 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-49e928d4-e1d5-4a9d-aef2-55b17d06f5db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:07:19.969332 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:19.986784 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 31/64] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:07:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:07:20.066708 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 34/65] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:07:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 09:07:20.271464 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccf6317-1fa6-4d91-ae5d-b52b9d22f3a4 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:20.389178 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e29873-def7-440a-9306-d737a44b0ab7 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:20.390002 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e29873-def7-440a-9306-d737a44b0ab7 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:20.390475 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e29873-def7-440a-9306-d737a44b0ab7 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:20.392014 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 35/66] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:07:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:07:20.404945 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c74c06f-ce5b-4f4a-ae6f-6d5bb7333f6a None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:20.405971 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c74c06f-ce5b-4f4a-ae6f-6d5bb7333f6a None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:20.406392 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c74c06f-ce5b-4f4a-ae6f-6d5bb7333f6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:20.416303 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccf6317-1fa6-4d91-ae5d-b52b9d22f3a4 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:20.440586 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-5c74c06f-ce5b-4f4a-ae6f-6d5bb7333f6a None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:07:20.442139 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-5c74c06f-ce5b-4f4a-ae6f-6d5bb7333f6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:07:20.453135 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c246fa04-1077-4777-8d12-b1945e6c0aa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:20.454057 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c246fa04-1077-4777-8d12-b1945e6c0aa5 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:20.454541 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c246fa04-1077-4777-8d12-b1945e6c0aa5 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:20.454987 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c246fa04-1077-4777-8d12-b1945e6c0aa5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:20.470333 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c246fa04-1077-4777-8d12-b1945e6c0aa5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:07:20.475481 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-c246fa04-1077-4777-8d12-b1945e6c0aa5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 09:07:20.475481 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:07:20.475481 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:07:20.475481 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:07:20.475481 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:07:20.475481 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:07:20.475481 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:07:20.475481 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:07:20.475481 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:07:20.475481 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:07:20.475481 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:07:20.475481 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:07:20.475481 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:07:20.475481 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:07:20.475481 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:07:20.475481 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:07:20.476383 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:07:20.476383 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:07:20.476383 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:07:20.476383 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:07:20.476383 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:07:20.476383 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:07:20.476383 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:07:20.476383 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:07:20.476383 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:07:20.476383 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:07:20.476383 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:07:20.476383 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 24 09:07:20.476383 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:07:20.476383 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:07:20.476383 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:07:20.476383 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:07:20.477159 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:07:20.477159 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:07:20.477159 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:07:20.477159 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:07:20.477159 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:07:20.477159 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:07:20.477159 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:07:20.477159 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:07:20.477159 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:07:20.477159 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:07:20.477159 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 09:07:20.477159 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 24 09:07:20.484005 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 32/67] 10.222.0.16 () {56 vars in 1099 bytes} [Wed Aug 24 09:07:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:07:20.488921 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c246fa04-1077-4777-8d12-b1945e6c0aa5 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:20.519255 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:20.543483 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 36/68] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:07:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:07:20.544373 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c74c06f-ce5b-4f4a-ae6f-6d5bb7333f6a None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:20.556822 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:20.609436 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c246fa04-1077-4777-8d12-b1945e6c0aa5 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:20.644705 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09db7105-d7a8-497f-973f-f4b04d95d275 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:20.645317 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09db7105-d7a8-497f-973f-f4b04d95d275 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:20.645609 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09db7105-d7a8-497f-973f-f4b04d95d275 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:20.645892 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09db7105-d7a8-497f-973f-f4b04d95d275 None admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:20.646565 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09db7105-d7a8-497f-973f-f4b04d95d275 None admin] RBAC: Adding query filter params (name=member) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:07:20.646878 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09db7105-d7a8-497f-973f-f4b04d95d275 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:20.648310 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09db7105-d7a8-497f-973f-f4b04d95d275 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:20.653612 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c74c06f-ce5b-4f4a-ae6f-6d5bb7333f6a None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:20.657450 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 33/69] 10.222.0.16 () {56 vars in 1101 bytes} [Wed Aug 24 09:07:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:07:20.658456 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746d6b8d-2210-4ace-87b5-9ecc140e3dfc None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:20.658744 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746d6b8d-2210-4ace-87b5-9ecc140e3dfc None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:20.659023 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746d6b8d-2210-4ace-87b5-9ecc140e3dfc None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:20.660552 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 34/70] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:07:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:07:20.661751 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc3cb40-912d-4a5d-8c89-b40bcda5d232 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:20.661853 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc3cb40-912d-4a5d-8c89-b40bcda5d232 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:20.662133 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc3cb40-912d-4a5d-8c89-b40bcda5d232 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:20.663455 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 35/71] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:07:20 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:07:20.664068 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc3cb40-912d-4a5d-8c89-b40bcda5d232 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:20.697415 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a20eb36-14f2-4329-b96e-13d74da5eae4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:20.698405 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a20eb36-14f2-4329-b96e-13d74da5eae4 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:20.698883 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a20eb36-14f2-4329-b96e-13d74da5eae4 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:20.699371 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a20eb36-14f2-4329-b96e-13d74da5eae4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:20.710059 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a20eb36-14f2-4329-b96e-13d74da5eae4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:07:20.714835 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-9a20eb36-14f2-4329-b96e-13d74da5eae4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:07:20.714835 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:07:20.714835 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:07:20.714835 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:07:20.714835 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:07:20.714835 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:07:20.714835 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:07:20.714835 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:07:20.714835 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:07:20.714835 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:07:20.714835 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:07:20.714835 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:07:20.714835 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:07:20.714835 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:07:20.714835 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:07:20.714835 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:07:20.715713 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:07:20.715713 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:07:20.715713 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:07:20.715713 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:07:20.715713 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:07:20.715713 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:07:20.715713 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:07:20.715713 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:07:20.715713 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:07:20.715713 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:07:20.715713 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:07:20.715713 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Aug 24 09:07:20.715713 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:07:20.715713 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:07:20.715713 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:07:20.715713 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:07:20.716467 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:07:20.716467 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:07:20.716467 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:07:20.716467 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:07:20.716467 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:07:20.716467 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:07:20.716467 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:07:20.716467 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:07:20.716467 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:07:20.716467 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:07:20.716467 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:07:20.716467 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Aug 24 09:07:20.716467 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 37/72] 10.222.0.16 () {56 vars in 1096 bytes} [Wed Aug 24 09:07:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:07:20.718216 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcc0d9a-ac5e-4c10-bceb-dc8a8ec0064c None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:20.718320 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcc0d9a-ac5e-4c10-bceb-dc8a8ec0064c None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:20.718577 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcc0d9a-ac5e-4c10-bceb-dc8a8ec0064c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:20.735927 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-afcc0d9a-ac5e-4c10-bceb-dc8a8ec0064c None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:07:20.737111 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-afcc0d9a-ac5e-4c10-bceb-dc8a8ec0064c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:07:20.768454 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:20.777232 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc3cb40-912d-4a5d-8c89-b40bcda5d232 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:20.795161 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:20.815777 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 38/73] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:07:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:07:20.826384 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a303381f-2244-451c-a9fa-f245959605d3 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:20.826384 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a303381f-2244-451c-a9fa-f245959605d3 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:20.826384 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a303381f-2244-451c-a9fa-f245959605d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:20.826655 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc939b6a-50c7-4139-923d-988728bc4d74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:20.826655 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc939b6a-50c7-4139-923d-988728bc4d74 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:20.826655 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc939b6a-50c7-4139-923d-988728bc4d74 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:20.826655 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc939b6a-50c7-4139-923d-988728bc4d74 None admin] PATH_INFO: `/v3/users/1925e7a247b64a06b5abece3c5687b12` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:20.839992 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc939b6a-50c7-4139-923d-988728bc4d74 None admin] RBAC: Authorizing `identity:get_user(user_id=1925e7a247b64a06b5abece3c5687b12)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:20.841491 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc939b6a-50c7-4139-923d-988728bc4d74 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:20.844026 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 36/74] 10.222.0.16 () {56 vars in 1177 bytes} [Wed Aug 24 09:07:20 2022] GET /identity/v3/users/1925e7a247b64a06b5abece3c5687b12 => generated 275 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:07:20.844667 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc939b6a-50c7-4139-923d-988728bc4d74 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:20.943361 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-a303381f-2244-451c-a9fa-f245959605d3 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:07:20.944597 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-a303381f-2244-451c-a9fa-f245959605d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:07:20.960525 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc939b6a-50c7-4139-923d-988728bc4d74 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:20.995631 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba3059f-25a4-403d-ba4e-ec03add807c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:20.996568 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba3059f-25a4-403d-ba4e-ec03add807c2 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:20.997056 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba3059f-25a4-403d-ba4e-ec03add807c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:20.997546 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba3059f-25a4-403d-ba4e-ec03add807c2 None admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:20.998413 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba3059f-25a4-403d-ba4e-ec03add807c2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:07:20.998920 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba3059f-25a4-403d-ba4e-ec03add807c2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:21.000630 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba3059f-25a4-403d-ba4e-ec03add807c2 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:21.009074 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 37/75] 10.222.0.16 () {56 vars in 1099 bytes} [Wed Aug 24 09:07:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:07:21.010110 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba3059f-25a4-403d-ba4e-ec03add807c2 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:21.019941 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 39/76] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:07:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:07:21.020997 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a303381f-2244-451c-a9fa-f245959605d3 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:21.055365 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:21.133448 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba3059f-25a4-403d-ba4e-ec03add807c2 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:21.133940 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:21.138301 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a303381f-2244-451c-a9fa-f245959605d3 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:21.172327 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3915203e-4278-44b0-8804-f707c51439d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:21.173108 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3915203e-4278-44b0-8804-f707c51439d1 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:21.173429 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3915203e-4278-44b0-8804-f707c51439d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:21.173706 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3915203e-4278-44b0-8804-f707c51439d1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:21.174204 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fda8593-c9a3-4a8d-90ef-76cc9a1f38b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:21.175138 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fda8593-c9a3-4a8d-90ef-76cc9a1f38b6 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:21.175427 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fda8593-c9a3-4a8d-90ef-76cc9a1f38b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:21.175742 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fda8593-c9a3-4a8d-90ef-76cc9a1f38b6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:21.185508 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3915203e-4278-44b0-8804-f707c51439d1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:07:21.190337 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-3915203e-4278-44b0-8804-f707c51439d1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 09:07:21.190337 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:07:21.190337 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:07:21.190337 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:07:21.190337 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:07:21.190337 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:07:21.190337 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:07:21.190337 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:07:21.190337 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:07:21.190337 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:07:21.190337 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:07:21.190337 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:07:21.190337 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:07:21.190337 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:07:21.190337 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:07:21.190337 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:07:21.191162 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:07:21.191162 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:07:21.191162 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:07:21.191162 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:07:21.191162 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:07:21.191162 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:07:21.191162 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:07:21.191162 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:07:21.191162 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:07:21.191162 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:07:21.191162 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:07:21.191162 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 24 09:07:21.191162 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:07:21.191162 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:07:21.191162 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:07:21.191162 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:07:21.192058 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:07:21.192058 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:07:21.192058 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:07:21.192058 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:07:21.192058 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:07:21.192058 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:07:21.192058 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:07:21.192058 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:07:21.192058 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:07:21.192058 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:07:21.192058 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 09:07:21.192058 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 24 09:07:21.192058 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 38/77] 10.222.0.16 () {56 vars in 1114 bytes} [Wed Aug 24 09:07:21 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:07:21.192688 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3915203e-4278-44b0-8804-f707c51439d1 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:21.255214 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fda8593-c9a3-4a8d-90ef-76cc9a1f38b6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:21.256974 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fda8593-c9a3-4a8d-90ef-76cc9a1f38b6 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:21.290476 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:21.300039 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3915203e-4278-44b0-8804-f707c51439d1 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:21.334983 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6262e035-95a8-482d-8f0e-010b50494dbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:21.335607 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6262e035-95a8-482d-8f0e-010b50494dbb None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:21.335916 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6262e035-95a8-482d-8f0e-010b50494dbb None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:21.336217 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6262e035-95a8-482d-8f0e-010b50494dbb None admin] PATH_INFO: `/v3/users/d1828412cb04465dbfa15b26582aa6de` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:21.337755 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6262e035-95a8-482d-8f0e-010b50494dbb None admin] RBAC: Authorizing `identity:get_user(user_id=d1828412cb04465dbfa15b26582aa6de)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:21.339211 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6262e035-95a8-482d-8f0e-010b50494dbb None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:21.340849 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 39/78] 10.222.0.16 () {56 vars in 1177 bytes} [Wed Aug 24 09:07:21 2022] GET /identity/v3/users/d1828412cb04465dbfa15b26582aa6de => generated 247 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:07:21.341616 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6262e035-95a8-482d-8f0e-010b50494dbb None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:21.357097 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:21.361498 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 40/79] 10.222.0.16 () {58 vars in 1282 bytes} [Wed Aug 24 09:07:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 09:07:21.362309 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fda8593-c9a3-4a8d-90ef-76cc9a1f38b6 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:21.442882 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6262e035-95a8-482d-8f0e-010b50494dbb None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:21.461558 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fda8593-c9a3-4a8d-90ef-76cc9a1f38b6 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:21.478270 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4655d21c-c245-465a-be1b-77e923169f5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:21.479091 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4655d21c-c245-465a-be1b-77e923169f5c None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:21.479375 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4655d21c-c245-465a-be1b-77e923169f5c None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:21.479683 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4655d21c-c245-465a-be1b-77e923169f5c None admin] PATH_INFO: `/v3/roles/member` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:21.489598 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4655d21c-c245-465a-be1b-77e923169f5c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:07:21.494353 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-4655d21c-c245-465a-be1b-77e923169f5c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 09:07:21.494353 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:07:21.494353 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:07:21.494353 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:07:21.494353 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:07:21.494353 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:07:21.494353 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:07:21.494353 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:07:21.494353 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:07:21.494353 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:07:21.494353 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:07:21.494353 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:07:21.494353 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:07:21.494353 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:07:21.494353 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:07:21.494353 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:07:21.495226 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:07:21.495226 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:07:21.495226 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:07:21.495226 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:07:21.495226 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:07:21.495226 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:07:21.495226 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:07:21.495226 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:07:21.495226 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:07:21.495226 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:07:21.495226 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:07:21.495226 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 24 09:07:21.495226 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:07:21.495226 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:07:21.495226 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:07:21.495226 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:07:21.499755 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:07:21.499755 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:07:21.499755 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:07:21.499755 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:07:21.499755 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:07:21.499755 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:07:21.499755 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:07:21.499755 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:07:21.499755 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:07:21.499755 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:07:21.499755 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 09:07:21.499755 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 24 09:07:21.500384 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08dcaac-8a23-4595-9528-b2a08d4618d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:21.500384 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08dcaac-8a23-4595-9528-b2a08d4618d0 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:21.500519 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 40/80] 10.222.0.16 () {56 vars in 1099 bytes} [Wed Aug 24 09:07:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:07:21.500581 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08dcaac-8a23-4595-9528-b2a08d4618d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:21.500581 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08dcaac-8a23-4595-9528-b2a08d4618d0 None admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:21.500701 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4655d21c-c245-465a-be1b-77e923169f5c None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:21.500765 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08dcaac-8a23-4595-9528-b2a08d4618d0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:07:21.500765 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08dcaac-8a23-4595-9528-b2a08d4618d0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:21.500765 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08dcaac-8a23-4595-9528-b2a08d4618d0 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:21.508987 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 41/81] 10.222.0.16 () {56 vars in 1111 bytes} [Wed Aug 24 09:07:21 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:07:21.509780 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08dcaac-8a23-4595-9528-b2a08d4618d0 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:21.522905 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:21.568045 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:21.599408 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4655d21c-c245-465a-be1b-77e923169f5c None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:21.608367 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08dcaac-8a23-4595-9528-b2a08d4618d0 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:21.632797 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2dbdcb-8ab2-4c08-b061-6c6af63406c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:21.633424 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2dbdcb-8ab2-4c08-b061-6c6af63406c3 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:21.633707 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2dbdcb-8ab2-4c08-b061-6c6af63406c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:21.633981 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2dbdcb-8ab2-4c08-b061-6c6af63406c3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:21.635050 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2dbdcb-8ab2-4c08-b061-6c6af63406c3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:21.636535 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2dbdcb-8ab2-4c08-b061-6c6af63406c3 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:21.638081 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 41/82] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 24 09:07:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:07:21.638843 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2dbdcb-8ab2-4c08-b061-6c6af63406c3 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:21.643445 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49444f57-f90f-45fa-a68b-a2ca99f532e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:21.644096 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49444f57-f90f-45fa-a68b-a2ca99f532e6 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:21.644385 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49444f57-f90f-45fa-a68b-a2ca99f532e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:21.644668 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49444f57-f90f-45fa-a68b-a2ca99f532e6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:21.712670 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49444f57-f90f-45fa-a68b-a2ca99f532e6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:21.714138 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49444f57-f90f-45fa-a68b-a2ca99f532e6 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:21.726988 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:21.738849 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2dbdcb-8ab2-4c08-b061-6c6af63406c3 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:21.771176 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:21.772699 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376cb492-6980-431c-8026-0c978737d6ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:21.773435 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376cb492-6980-431c-8026-0c978737d6ef None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:21.773719 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376cb492-6980-431c-8026-0c978737d6ef None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:21.773997 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376cb492-6980-431c-8026-0c978737d6ef None admin] PATH_INFO: `/v3/projects/7d6be68cf8b046d5b03fba3c5a367c11` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:21.775109 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376cb492-6980-431c-8026-0c978737d6ef None admin] RBAC: Authorizing `identity:get_project(project_id=7d6be68cf8b046d5b03fba3c5a367c11)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:21.776575 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376cb492-6980-431c-8026-0c978737d6ef None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:21.778187 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 42/83] 10.222.0.16 () {56 vars in 1186 bytes} [Wed Aug 24 09:07:21 2022] GET /identity/v3/projects/7d6be68cf8b046d5b03fba3c5a367c11 => generated 298 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:07:21.778978 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376cb492-6980-431c-8026-0c978737d6ef None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:21.819556 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 42/84] 10.222.0.16 () {58 vars in 1282 bytes} [Wed Aug 24 09:07:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 09:07:21.820426 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49444f57-f90f-45fa-a68b-a2ca99f532e6 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:21.890001 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376cb492-6980-431c-8026-0c978737d6ef None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:21.918681 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49444f57-f90f-45fa-a68b-a2ca99f532e6 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:21.923494 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e844f00-6675-4d21-8bc2-0243b17718e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:21.924124 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e844f00-6675-4d21-8bc2-0243b17718e3 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:21.924404 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e844f00-6675-4d21-8bc2-0243b17718e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:21.924676 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e844f00-6675-4d21-8bc2-0243b17718e3 None admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:21.925334 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e844f00-6675-4d21-8bc2-0243b17718e3 None admin] RBAC: Adding query filter params (name=member) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:07:21.925636 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e844f00-6675-4d21-8bc2-0243b17718e3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:21.927005 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e844f00-6675-4d21-8bc2-0243b17718e3 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:21.934751 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 43/85] 10.222.0.16 () {56 vars in 1101 bytes} [Wed Aug 24 09:07:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:07:21.935513 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e844f00-6675-4d21-8bc2-0243b17718e3 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:21.952802 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928a0183-3f12-4094-b35b-5d6e2c763077 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:21.953439 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928a0183-3f12-4094-b35b-5d6e2c763077 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:21.953725 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928a0183-3f12-4094-b35b-5d6e2c763077 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:21.954002 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928a0183-3f12-4094-b35b-5d6e2c763077 None admin] PATH_INFO: `/v3/users/1925e7a247b64a06b5abece3c5687b12` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:21.955657 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928a0183-3f12-4094-b35b-5d6e2c763077 None admin] RBAC: Authorizing `identity:get_user(user_id=1925e7a247b64a06b5abece3c5687b12)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:21.956231 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:21.957276 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928a0183-3f12-4094-b35b-5d6e2c763077 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:21.958678 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 43/86] 10.222.0.16 () {56 vars in 1177 bytes} [Wed Aug 24 09:07:21 2022] GET /identity/v3/users/1925e7a247b64a06b5abece3c5687b12 => generated 275 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:07:21.959442 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928a0183-3f12-4094-b35b-5d6e2c763077 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:21.988741 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:22.039122 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e844f00-6675-4d21-8bc2-0243b17718e3 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:22.058555 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928a0183-3f12-4094-b35b-5d6e2c763077 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:22.072709 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d5bc11-d709-4dd7-9ed6-cb30678648d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:22.073316 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d5bc11-d709-4dd7-9ed6-cb30678648d0 None admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:22.073597 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d5bc11-d709-4dd7-9ed6-cb30678648d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:22.073871 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d5bc11-d709-4dd7-9ed6-cb30678648d0 None admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:22.074509 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d5bc11-d709-4dd7-9ed6-cb30678648d0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:22.075967 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d5bc11-d709-4dd7-9ed6-cb30678648d0 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:22.091947 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd798f87-3fa2-4ecb-81b6-0bb46a6684a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:22.093085 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd798f87-3fa2-4ecb-81b6-0bb46a6684a3 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:22.093362 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd798f87-3fa2-4ecb-81b6-0bb46a6684a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:22.093631 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd798f87-3fa2-4ecb-81b6-0bb46a6684a3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:22.094349 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd798f87-3fa2-4ecb-81b6-0bb46a6684a3 None admin] RBAC: Adding query filter params (role.id=136cdf78e00a426997dd4e37faff489b, scope.project.id=7d6be68cf8b046d5b03fba3c5a367c11, user.id=1925e7a247b64a06b5abece3c5687b12) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:07:22.094655 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd798f87-3fa2-4ecb-81b6-0bb46a6684a3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:22.096218 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd798f87-3fa2-4ecb-81b6-0bb46a6684a3 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:22.101481 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 44/87] 10.222.0.16 () {56 vars in 1374 bytes} [Wed Aug 24 09:07:21 2022] GET /identity/v3/role_assignments?user.id=1925e7a247b64a06b5abece3c5687b12&scope.project.id=7d6be68cf8b046d5b03fba3c5a367c11&role.id=136cdf78e00a426997dd4e37faff489b => generated 259 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:07:22.102258 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd798f87-3fa2-4ecb-81b6-0bb46a6684a3 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:22.204518 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:22.213453 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd798f87-3fa2-4ecb-81b6-0bb46a6684a3 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:22.249079 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026db0dd-8d00-44c5-a1a5-89098d2705e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:22.249351 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026db0dd-8d00-44c5-a1a5-89098d2705e6 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:22.249700 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026db0dd-8d00-44c5-a1a5-89098d2705e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:22.250145 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026db0dd-8d00-44c5-a1a5-89098d2705e6 None admin] PATH_INFO: `/v3/projects/7d6be68cf8b046d5b03fba3c5a367c11` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:22.251533 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026db0dd-8d00-44c5-a1a5-89098d2705e6 None admin] RBAC: Authorizing `identity:get_project(project_id=7d6be68cf8b046d5b03fba3c5a367c11)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:22.253013 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026db0dd-8d00-44c5-a1a5-89098d2705e6 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:22.255297 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 45/88] 10.222.0.16 () {56 vars in 1186 bytes} [Wed Aug 24 09:07:22 2022] GET /identity/v3/projects/7d6be68cf8b046d5b03fba3c5a367c11 => generated 298 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:07:22.255455 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026db0dd-8d00-44c5-a1a5-89098d2705e6 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:22.256816 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 44/89] 10.222.0.16 () {60 vars in 1132 bytes} [Wed Aug 24 09:07:21 2022] POST /identity/v3/users => generated 289 bytes in 321 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:07:22.257554 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d5bc11-d709-4dd7-9ed6-cb30678648d0 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:22.373313 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:22.378855 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026db0dd-8d00-44c5-a1a5-89098d2705e6 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:22.385340 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d5bc11-d709-4dd7-9ed6-cb30678648d0 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:22.422987 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fa5cf5-83cb-409b-a608-0eac1a5e7d84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:22.423912 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fa5cf5-83cb-409b-a608-0eac1a5e7d84 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:22.424338 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fa5cf5-83cb-409b-a608-0eac1a5e7d84 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:22.424804 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fa5cf5-83cb-409b-a608-0eac1a5e7d84 None admin] PATH_INFO: `/v3/users/1925e7a247b64a06b5abece3c5687b12` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:22.426154 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fa5cf5-83cb-409b-a608-0eac1a5e7d84 None admin] RBAC: Authorizing `identity:get_user(user_id=1925e7a247b64a06b5abece3c5687b12)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:22.428022 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fa5cf5-83cb-409b-a608-0eac1a5e7d84 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:22.429074 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ba5836-10db-40b7-93c9-99678a271aab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:22.429699 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ba5836-10db-40b7-93c9-99678a271aab None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:22.429986 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ba5836-10db-40b7-93c9-99678a271aab None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:22.430233 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 46/90] 10.222.0.16 () {56 vars in 1177 bytes} [Wed Aug 24 09:07:22 2022] GET /identity/v3/users/1925e7a247b64a06b5abece3c5687b12 => generated 275 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:07:22.431573 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ba5836-10db-40b7-93c9-99678a271aab None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:22.431750 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fa5cf5-83cb-409b-a608-0eac1a5e7d84 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:22.473047 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:22.533966 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ba5836-10db-40b7-93c9-99678a271aab None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:22.535851 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ba5836-10db-40b7-93c9-99678a271aab None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:22.576842 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fa5cf5-83cb-409b-a608-0eac1a5e7d84 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:22.621806 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb014bda-b242-4d7d-8e3c-37f0ec2789a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:22.622439 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb014bda-b242-4d7d-8e3c-37f0ec2789a1 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:22.622726 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb014bda-b242-4d7d-8e3c-37f0ec2789a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:22.623004 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb014bda-b242-4d7d-8e3c-37f0ec2789a1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:22.624358 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb014bda-b242-4d7d-8e3c-37f0ec2789a1 None admin] RBAC: Adding query filter params (role.id=15f634d4a2d247ddab90c458e5e1c141, scope.project.id=7d6be68cf8b046d5b03fba3c5a367c11, user.id=d1828412cb04465dbfa15b26582aa6de) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:07:22.624715 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb014bda-b242-4d7d-8e3c-37f0ec2789a1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:22.626163 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb014bda-b242-4d7d-8e3c-37f0ec2789a1 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:22.632854 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 47/91] 10.222.0.16 () {56 vars in 1374 bytes} [Wed Aug 24 09:07:22 2022] GET /identity/v3/role_assignments?user.id=d1828412cb04465dbfa15b26582aa6de&scope.project.id=7d6be68cf8b046d5b03fba3c5a367c11&role.id=15f634d4a2d247ddab90c458e5e1c141 => generated 259 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:07:22.633681 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb014bda-b242-4d7d-8e3c-37f0ec2789a1 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:22.637624 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:22.667344 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 45/92] 10.222.0.16 () {58 vars in 1282 bytes} [Wed Aug 24 09:07:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 09:07:22.674246 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ba5836-10db-40b7-93c9-99678a271aab None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:22.798533 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:22.818457 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb014bda-b242-4d7d-8e3c-37f0ec2789a1 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:22.863152 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ba5836-10db-40b7-93c9-99678a271aab None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:22.883301 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22835a04-d12e-4360-ad92-abeadf15de8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:22.883945 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22835a04-d12e-4360-ad92-abeadf15de8d None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:22.884229 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22835a04-d12e-4360-ad92-abeadf15de8d None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:22.884502 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22835a04-d12e-4360-ad92-abeadf15de8d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:22.926968 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9358cfe-1bf4-497a-9709-1082301b1e10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:22.927695 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9358cfe-1bf4-497a-9709-1082301b1e10 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:22.927977 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9358cfe-1bf4-497a-9709-1082301b1e10 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:22.928248 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9358cfe-1bf4-497a-9709-1082301b1e10 None admin] PATH_INFO: `/v3/projects/7d6be68cf8b046d5b03fba3c5a367c11` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:22.929608 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9358cfe-1bf4-497a-9709-1082301b1e10 None admin] RBAC: Authorizing `identity:get_project(project_id=7d6be68cf8b046d5b03fba3c5a367c11)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:22.932338 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9358cfe-1bf4-497a-9709-1082301b1e10 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:22.934000 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 46/93] 10.222.0.16 () {56 vars in 1186 bytes} [Wed Aug 24 09:07:22 2022] GET /identity/v3/projects/7d6be68cf8b046d5b03fba3c5a367c11 => generated 298 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:07:22.945457 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9358cfe-1bf4-497a-9709-1082301b1e10 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:23.021104 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22835a04-d12e-4360-ad92-abeadf15de8d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:23.023794 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22835a04-d12e-4360-ad92-abeadf15de8d None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:23.028483 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:23.110868 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9358cfe-1bf4-497a-9709-1082301b1e10 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:23.169152 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:23.179859 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 48/94] 10.222.0.16 () {58 vars in 1282 bytes} [Wed Aug 24 09:07:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 546 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 09:07:23.187155 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22835a04-d12e-4360-ad92-abeadf15de8d None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:23.208311 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff80617-e139-4e9f-9816-5342ecdc6538 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:23.208945 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff80617-e139-4e9f-9816-5342ecdc6538 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:23.209231 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff80617-e139-4e9f-9816-5342ecdc6538 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:23.209515 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff80617-e139-4e9f-9816-5342ecdc6538 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:23.210272 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff80617-e139-4e9f-9816-5342ecdc6538 None admin] RBAC: Adding query filter params (role.id=ed6466d17fb7403c8491bb7f1ef95fc1, scope.project.id=7d6be68cf8b046d5b03fba3c5a367c11, user.id=1925e7a247b64a06b5abece3c5687b12) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:07:23.210614 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff80617-e139-4e9f-9816-5342ecdc6538 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:23.212100 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff80617-e139-4e9f-9816-5342ecdc6538 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:23.232867 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 47/95] 10.222.0.16 () {56 vars in 1374 bytes} [Wed Aug 24 09:07:22 2022] GET /identity/v3/role_assignments?user.id=1925e7a247b64a06b5abece3c5687b12&scope.project.id=7d6be68cf8b046d5b03fba3c5a367c11&role.id=ed6466d17fb7403c8491bb7f1ef95fc1 => generated 259 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:07:23.319805 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22835a04-d12e-4360-ad92-abeadf15de8d None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:23.359016 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb36faa6-99c5-42d5-8de5-6726a541bb9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:23.359997 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb36faa6-99c5-42d5-8de5-6726a541bb9b None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:23.360423 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb36faa6-99c5-42d5-8de5-6726a541bb9b None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:23.360841 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb36faa6-99c5-42d5-8de5-6726a541bb9b None admin] PATH_INFO: `/v3/projects/d6829a2bfe5a433dbc0b7b3dc0283fdf` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:23.362372 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb36faa6-99c5-42d5-8de5-6726a541bb9b None admin] RBAC: Authorizing `identity:get_project(project_id=d6829a2bfe5a433dbc0b7b3dc0283fdf)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:23.364076 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb36faa6-99c5-42d5-8de5-6726a541bb9b None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:23.366001 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 49/96] 10.222.0.16 () {56 vars in 1186 bytes} [Wed Aug 24 09:07:23 2022] GET /identity/v3/projects/d6829a2bfe5a433dbc0b7b3dc0283fdf => generated 312 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:07:23.372424 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff80617-e139-4e9f-9816-5342ecdc6538 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:23.503244 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff80617-e139-4e9f-9816-5342ecdc6538 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:23.510856 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:23.542619 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a095c72-b979-4884-98aa-07a89307d6e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:23.543483 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a095c72-b979-4884-98aa-07a89307d6e3 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:23.544012 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a095c72-b979-4884-98aa-07a89307d6e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:23.544429 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a095c72-b979-4884-98aa-07a89307d6e3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:23.545310 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a095c72-b979-4884-98aa-07a89307d6e3 None admin] RBAC: Adding query filter params (role.id=136cdf78e00a426997dd4e37faff489b, scope.project.id=d6829a2bfe5a433dbc0b7b3dc0283fdf, user.id=1925e7a247b64a06b5abece3c5687b12) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:07:23.546478 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a095c72-b979-4884-98aa-07a89307d6e3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:23.548112 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a095c72-b979-4884-98aa-07a89307d6e3 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:23.560840 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 48/97] 10.222.0.16 () {56 vars in 1374 bytes} [Wed Aug 24 09:07:23 2022] GET /identity/v3/role_assignments?user.id=1925e7a247b64a06b5abece3c5687b12&scope.project.id=d6829a2bfe5a433dbc0b7b3dc0283fdf&role.id=136cdf78e00a426997dd4e37faff489b => generated 259 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:07:24.457816 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d86ae3-3119-4297-9435-9378f6d83370 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:24.458356 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d86ae3-3119-4297-9435-9378f6d83370 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:24.458356 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d86ae3-3119-4297-9435-9378f6d83370 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:24.459970 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 50/98] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:07:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:07:24.465300 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb15298-48fe-4981-a9a7-7e517d609d5c None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:24.465697 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb15298-48fe-4981-a9a7-7e517d609d5c None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:24.465859 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb15298-48fe-4981-a9a7-7e517d609d5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:24.487715 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-5eb15298-48fe-4981-a9a7-7e517d609d5c None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:07:24.488898 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-5eb15298-48fe-4981-a9a7-7e517d609d5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:07:24.582247 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 49/99] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:07:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:07:25.125367 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d86ae3-3119-4297-9435-9378f6d83370 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:25.207858 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:25.268456 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d86ae3-3119-4297-9435-9378f6d83370 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:25.271805 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93b0dcb-d8ab-446a-9ba8-16f2ce6aa9f1 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:25.272102 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93b0dcb-d8ab-446a-9ba8-16f2ce6aa9f1 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:25.272447 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93b0dcb-d8ab-446a-9ba8-16f2ce6aa9f1 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:25.274052 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 50/100] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:07:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:07:25.278893 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da518d39-6a6f-417e-9876-cac68dbe8db1 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:25.279181 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da518d39-6a6f-417e-9876-cac68dbe8db1 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:25.279463 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da518d39-6a6f-417e-9876-cac68dbe8db1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:25.329022 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-da518d39-6a6f-417e-9876-cac68dbe8db1 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:07:25.330440 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-da518d39-6a6f-417e-9876-cac68dbe8db1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:07:25.359016 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d643e1-960c-4724-87f8-7d9b90a24265 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:25.359683 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d643e1-960c-4724-87f8-7d9b90a24265 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:25.359974 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d643e1-960c-4724-87f8-7d9b90a24265 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:25.360254 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d643e1-960c-4724-87f8-7d9b90a24265 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:25.374982 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d643e1-960c-4724-87f8-7d9b90a24265 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:07:25.380380 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-f5d643e1-960c-4724-87f8-7d9b90a24265 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 09:07:25.380380 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:07:25.380380 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:07:25.380380 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:07:25.380380 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:07:25.380380 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:07:25.380380 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:07:25.380380 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:07:25.380380 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:07:25.380380 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:07:25.380380 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:07:25.380380 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:07:25.380380 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:07:25.380380 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:07:25.380380 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:07:25.380380 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:07:25.381209 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:07:25.381209 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:07:25.381209 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:07:25.381209 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:07:25.381209 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:07:25.381209 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:07:25.381209 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:07:25.381209 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:07:25.381209 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:07:25.381209 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:07:25.381209 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:07:25.381209 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Aug 24 09:07:25.381209 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:07:25.381209 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:07:25.381209 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:07:25.381209 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:07:25.381983 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:07:25.381983 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:07:25.381983 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:07:25.381983 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:07:25.381983 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:07:25.381983 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:07:25.381983 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:07:25.381983 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:07:25.381983 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:07:25.381983 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:07:25.381983 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 09:07:25.381983 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Aug 24 09:07:25.385066 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 51/101] 10.222.0.16 () {56 vars in 1099 bytes} [Wed Aug 24 09:07:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:07:25.387170 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8387129e-783b-4fc3-979c-3d0e0f30bebe None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:25.387689 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8387129e-783b-4fc3-979c-3d0e0f30bebe None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:25.388197 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8387129e-783b-4fc3-979c-3d0e0f30bebe None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:25.390857 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 52/102] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:07:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:07:25.391806 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8387129e-783b-4fc3-979c-3d0e0f30bebe None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:25.504555 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 51/103] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:07:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:07:25.506272 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f266d9-2933-457c-b83c-d6d77db719b4 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:25.506571 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f266d9-2933-457c-b83c-d6d77db719b4 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:25.506845 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f266d9-2933-457c-b83c-d6d77db719b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:25.513482 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8387129e-783b-4fc3-979c-3d0e0f30bebe None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:25.526940 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-59f266d9-2933-457c-b83c-d6d77db719b4 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:07:25.528172 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-59f266d9-2933-457c-b83c-d6d77db719b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:07:25.545934 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:25.560411 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:25.563752 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40b82b5-ef05-493c-93a3-f956b106a858 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:25.564922 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40b82b5-ef05-493c-93a3-f956b106a858 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:25.565659 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40b82b5-ef05-493c-93a3-f956b106a858 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:25.566159 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40b82b5-ef05-493c-93a3-f956b106a858 None admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:25.567513 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40b82b5-ef05-493c-93a3-f956b106a858 None admin] RBAC: Adding query filter params (name=member) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:07:25.568365 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40b82b5-ef05-493c-93a3-f956b106a858 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:25.569972 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40b82b5-ef05-493c-93a3-f956b106a858 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:25.580479 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 53/104] 10.222.0.16 () {56 vars in 1101 bytes} [Wed Aug 24 09:07:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:07:25.582009 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c570db-e5ed-4538-930e-d57622194ead None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:25.582528 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c570db-e5ed-4538-930e-d57622194ead None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:25.583409 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c570db-e5ed-4538-930e-d57622194ead None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:25.585543 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 54/105] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:07:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:07:25.586460 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c570db-e5ed-4538-930e-d57622194ead None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:25.617438 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 52/106] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:07:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:07:25.619081 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8139d945-efb9-418e-a610-085275149263 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:25.619373 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8139d945-efb9-418e-a610-085275149263 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:25.619674 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8139d945-efb9-418e-a610-085275149263 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:25.643816 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-8139d945-efb9-418e-a610-085275149263 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:07:25.645690 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-8139d945-efb9-418e-a610-085275149263 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:07:25.760782 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c570db-e5ed-4538-930e-d57622194ead None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:25.782573 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 53/107] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:07:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:07:25.812081 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3678f7d0-ffe9-4abd-b5b7-6c9a8364aa1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:25.812732 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3678f7d0-ffe9-4abd-b5b7-6c9a8364aa1e None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:25.813018 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3678f7d0-ffe9-4abd-b5b7-6c9a8364aa1e None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:25.813296 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3678f7d0-ffe9-4abd-b5b7-6c9a8364aa1e None admin] PATH_INFO: `/v3/users/1925e7a247b64a06b5abece3c5687b12` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:25.814437 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3678f7d0-ffe9-4abd-b5b7-6c9a8364aa1e None admin] RBAC: Authorizing `identity:get_user(user_id=1925e7a247b64a06b5abece3c5687b12)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:25.815909 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3678f7d0-ffe9-4abd-b5b7-6c9a8364aa1e None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:25.817984 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 55/108] 10.222.0.16 () {56 vars in 1177 bytes} [Wed Aug 24 09:07:25 2022] GET /identity/v3/users/1925e7a247b64a06b5abece3c5687b12 => generated 275 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:07:25.823519 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8139d945-efb9-418e-a610-085275149263 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:25.872433 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3678f7d0-ffe9-4abd-b5b7-6c9a8364aa1e None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:25.943873 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8139d945-efb9-418e-a610-085275149263 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:25.953894 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:25.955946 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:25.984154 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858e05c1-e4dc-4cd8-8f9b-1434144735cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:25.984813 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858e05c1-e4dc-4cd8-8f9b-1434144735cb None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:25.985099 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858e05c1-e4dc-4cd8-8f9b-1434144735cb None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:25.985380 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858e05c1-e4dc-4cd8-8f9b-1434144735cb None admin] PATH_INFO: `/v3/projects/7d6be68cf8b046d5b03fba3c5a367c11` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:25.990152 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858e05c1-e4dc-4cd8-8f9b-1434144735cb None admin] RBAC: Authorizing `identity:get_project(project_id=7d6be68cf8b046d5b03fba3c5a367c11)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:25.992420 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858e05c1-e4dc-4cd8-8f9b-1434144735cb None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:25.994123 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 54/109] 10.222.0.16 () {56 vars in 1186 bytes} [Wed Aug 24 09:07:25 2022] GET /identity/v3/projects/7d6be68cf8b046d5b03fba3c5a367c11 => generated 298 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:07:26.000456 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858e05c1-e4dc-4cd8-8f9b-1434144735cb None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:26.049814 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3678f7d0-ffe9-4abd-b5b7-6c9a8364aa1e None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:26.090432 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b7630e-b9d2-4084-9f3c-a403f6a3fb06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:26.091078 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b7630e-b9d2-4084-9f3c-a403f6a3fb06 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:26.091392 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b7630e-b9d2-4084-9f3c-a403f6a3fb06 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:26.092010 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b7630e-b9d2-4084-9f3c-a403f6a3fb06 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:26.104133 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b7630e-b9d2-4084-9f3c-a403f6a3fb06 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:07:26.116517 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-17b7630e-b9d2-4084-9f3c-a403f6a3fb06 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 09:07:26.116517 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:07:26.116517 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:07:26.116517 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:07:26.116517 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:07:26.116517 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:07:26.116517 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:07:26.116517 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:07:26.116517 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:07:26.116517 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:07:26.116517 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:07:26.116517 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:07:26.116517 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:07:26.116517 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:07:26.116517 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:07:26.116517 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:07:26.117411 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:07:26.117411 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:07:26.117411 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:07:26.117411 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:07:26.117411 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:07:26.117411 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:07:26.117411 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:07:26.117411 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:07:26.117411 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:07:26.117411 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:07:26.117411 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:07:26.117411 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Aug 24 09:07:26.117411 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:07:26.117411 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:07:26.117411 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:07:26.117411 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:07:26.118169 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:07:26.118169 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:07:26.118169 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:07:26.118169 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:07:26.118169 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:07:26.118169 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:07:26.118169 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:07:26.118169 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:07:26.118169 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:07:26.118169 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:07:26.118169 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 09:07:26.118169 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Aug 24 09:07:26.118169 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 56/110] 10.222.0.16 () {56 vars in 1099 bytes} [Wed Aug 24 09:07:25 2022] GET /identity/v3/roles/reader => generated 84 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 09:07:26.118807 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b7630e-b9d2-4084-9f3c-a403f6a3fb06 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:26.178920 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858e05c1-e4dc-4cd8-8f9b-1434144735cb None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:26.219095 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4e6f1d-511b-4244-8e60-275ceb43d97e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:26.220216 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4e6f1d-511b-4244-8e60-275ceb43d97e None admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:26.220829 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4e6f1d-511b-4244-8e60-275ceb43d97e None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:26.221334 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4e6f1d-511b-4244-8e60-275ceb43d97e None admin] PATH_INFO: `/v3/projects/7d6be68cf8b046d5b03fba3c5a367c11/users/1925e7a247b64a06b5abece3c5687b12/roles/136cdf78e00a426997dd4e37faff489b` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:26.223758 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4e6f1d-511b-4244-8e60-275ceb43d97e None admin] RBAC: Authorizing `identity:create_grant(project_id=7d6be68cf8b046d5b03fba3c5a367c11, user_id=1925e7a247b64a06b5abece3c5687b12, role_id=136cdf78e00a426997dd4e37faff489b)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:26.225473 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4e6f1d-511b-4244-8e60-275ceb43d97e None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:26.246641 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b7630e-b9d2-4084-9f3c-a403f6a3fb06 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:26.258455 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:26.281608 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08efd79a-6841-437b-8b83-916564a7a4c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:26.282278 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08efd79a-6841-437b-8b83-916564a7a4c9 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:26.282568 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08efd79a-6841-437b-8b83-916564a7a4c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:26.282847 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08efd79a-6841-437b-8b83-916564a7a4c9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:26.293582 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08efd79a-6841-437b-8b83-916564a7a4c9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:07:26.299832 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-08efd79a-6841-437b-8b83-916564a7a4c9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:07:26.299832 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:07:26.299832 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:07:26.299832 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:07:26.299832 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:07:26.299832 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:07:26.299832 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:07:26.299832 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:07:26.299832 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:07:26.299832 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:07:26.299832 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:07:26.299832 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:07:26.299832 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:07:26.299832 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:07:26.299832 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:07:26.299832 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:07:26.300719 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:07:26.300719 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:07:26.300719 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:07:26.300719 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:07:26.300719 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:07:26.300719 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:07:26.300719 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:07:26.300719 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:07:26.300719 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:07:26.300719 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:07:26.300719 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:07:26.300719 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Aug 24 09:07:26.300719 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:07:26.300719 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:07:26.300719 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:07:26.300719 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:07:26.301514 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:07:26.301514 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:07:26.301514 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:07:26.301514 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:07:26.301514 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:07:26.301514 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:07:26.301514 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:07:26.301514 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:07:26.301514 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:07:26.301514 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:07:26.301514 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:07:26.301514 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Aug 24 09:07:26.301514 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 57/111] 10.222.0.16 () {56 vars in 1096 bytes} [Wed Aug 24 09:07:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:07:26.301514 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08efd79a-6841-437b-8b83-916564a7a4c9 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:26.318975 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 55/112] 10.222.0.16 () {58 vars in 1439 bytes} [Wed Aug 24 09:07:25 2022] PUT /identity/v3/projects/7d6be68cf8b046d5b03fba3c5a367c11/users/1925e7a247b64a06b5abece3c5687b12/roles/136cdf78e00a426997dd4e37faff489b => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:07:26.320997 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4e6f1d-511b-4244-8e60-275ceb43d97e None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:26.349405 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:26.463945 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4e6f1d-511b-4244-8e60-275ceb43d97e None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:26.487949 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08efd79a-6841-437b-8b83-916564a7a4c9 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:26.508560 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a66d6b-90cf-4b5e-bd80-b911513384e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:26.509206 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a66d6b-90cf-4b5e-bd80-b911513384e0 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:26.509478 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a66d6b-90cf-4b5e-bd80-b911513384e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:26.509783 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a66d6b-90cf-4b5e-bd80-b911513384e0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:26.510917 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a66d6b-90cf-4b5e-bd80-b911513384e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:26.512462 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a66d6b-90cf-4b5e-bd80-b911513384e0 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:26.514531 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 56/113] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 24 09:07:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:07:26.515453 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a66d6b-90cf-4b5e-bd80-b911513384e0 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:26.533188 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4f5632-1f52-462b-a2b8-9c0fbfbd93cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:26.533812 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4f5632-1f52-462b-a2b8-9c0fbfbd93cc None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:26.534099 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4f5632-1f52-462b-a2b8-9c0fbfbd93cc None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:26.534371 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4f5632-1f52-462b-a2b8-9c0fbfbd93cc None admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:26.535049 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4f5632-1f52-462b-a2b8-9c0fbfbd93cc None admin] RBAC: Adding query filter params (name=reader) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:07:26.535356 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4f5632-1f52-462b-a2b8-9c0fbfbd93cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:26.536801 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4f5632-1f52-462b-a2b8-9c0fbfbd93cc None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:26.545734 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 58/114] 10.222.0.16 () {56 vars in 1101 bytes} [Wed Aug 24 09:07:26 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:07:26.546914 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17055533-2103-41a9-be3e-c8fd546c70ee None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:26.547192 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17055533-2103-41a9-be3e-c8fd546c70ee None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:26.547472 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17055533-2103-41a9-be3e-c8fd546c70ee None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:26.553137 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 59/115] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:07:26 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:07:26.556794 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8203d8-66f6-4bbe-a3c9-061db7f85e17 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:26.557226 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8203d8-66f6-4bbe-a3c9-061db7f85e17 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:26.557564 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8203d8-66f6-4bbe-a3c9-061db7f85e17 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:26.558828 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 60/116] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:07:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:07:26.559516 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8203d8-66f6-4bbe-a3c9-061db7f85e17 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:26.596830 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:26.618361 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:26.638044 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a66d6b-90cf-4b5e-bd80-b911513384e0 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:26.669094 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8203d8-66f6-4bbe-a3c9-061db7f85e17 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:26.678744 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a836e8ad-45fc-457f-bf59-c5b803cf514b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:26.679361 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a836e8ad-45fc-457f-bf59-c5b803cf514b None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:26.679673 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a836e8ad-45fc-457f-bf59-c5b803cf514b None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:26.679956 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a836e8ad-45fc-457f-bf59-c5b803cf514b None admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:26.680808 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a836e8ad-45fc-457f-bf59-c5b803cf514b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:07:26.680913 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a836e8ad-45fc-457f-bf59-c5b803cf514b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:26.682273 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a836e8ad-45fc-457f-bf59-c5b803cf514b None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:26.691492 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 57/117] 10.222.0.16 () {56 vars in 1099 bytes} [Wed Aug 24 09:07:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:07:26.696429 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a836e8ad-45fc-457f-bf59-c5b803cf514b None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:26.708871 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6012bf2-1f52-4176-b5d1-c9338f7f688a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:26.709866 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6012bf2-1f52-4176-b5d1-c9338f7f688a None admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:26.710114 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6012bf2-1f52-4176-b5d1-c9338f7f688a None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:26.710338 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6012bf2-1f52-4176-b5d1-c9338f7f688a None admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:26.710971 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6012bf2-1f52-4176-b5d1-c9338f7f688a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:26.712449 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6012bf2-1f52-4176-b5d1-c9338f7f688a None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:26.777385 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 61/118] 10.222.0.16 () {60 vars in 1141 bytes} [Wed Aug 24 09:07:26 2022] POST /identity/v3/projects => generated 302 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:07:26.778971 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e625cc-34ca-45d4-9fcc-eb175eea8b4a None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:26.779518 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e625cc-34ca-45d4-9fcc-eb175eea8b4a None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:26.779518 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e625cc-34ca-45d4-9fcc-eb175eea8b4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:26.803626 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-20e625cc-34ca-45d4-9fcc-eb175eea8b4a None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:07:26.803814 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-20e625cc-34ca-45d4-9fcc-eb175eea8b4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:07:26.818025 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a836e8ad-45fc-457f-bf59-c5b803cf514b None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:26.870639 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db3d24d-66f1-41a6-9dc7-7163e8db2532 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:26.871317 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db3d24d-66f1-41a6-9dc7-7163e8db2532 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:26.871629 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db3d24d-66f1-41a6-9dc7-7163e8db2532 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:26.872213 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db3d24d-66f1-41a6-9dc7-7163e8db2532 None admin] PATH_INFO: `/v3/users/693fc8b7114049948366081c3e04537c` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:26.886704 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 62/119] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:07:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:07:26.888241 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ea09c8-5bfa-47cf-8b4e-50d52c45d0bf None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:26.888530 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ea09c8-5bfa-47cf-8b4e-50d52c45d0bf None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:26.888805 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ea09c8-5bfa-47cf-8b4e-50d52c45d0bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:26.891952 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db3d24d-66f1-41a6-9dc7-7163e8db2532 None admin] RBAC: Authorizing `identity:get_user(user_id=693fc8b7114049948366081c3e04537c)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:26.893640 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db3d24d-66f1-41a6-9dc7-7163e8db2532 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:26.895510 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 58/120] 10.222.0.16 () {56 vars in 1177 bytes} [Wed Aug 24 09:07:26 2022] GET /identity/v3/users/693fc8b7114049948366081c3e04537c => generated 289 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:07:26.902523 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db3d24d-66f1-41a6-9dc7-7163e8db2532 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:26.906918 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:26.910381 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-f6ea09c8-5bfa-47cf-8b4e-50d52c45d0bf None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:07:26.911955 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-f6ea09c8-5bfa-47cf-8b4e-50d52c45d0bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:07:26.925125 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:26.990852 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 63/121] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:07:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:07:26.991802 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ea09c8-5bfa-47cf-8b4e-50d52c45d0bf None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:27.019031 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db3d24d-66f1-41a6-9dc7-7163e8db2532 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:27.057074 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d798ed89-c16f-4f81-ae38-f7ffdf7f1287 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:27.057628 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d798ed89-c16f-4f81-ae38-f7ffdf7f1287 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:27.057916 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d798ed89-c16f-4f81-ae38-f7ffdf7f1287 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:27.058195 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d798ed89-c16f-4f81-ae38-f7ffdf7f1287 None admin] PATH_INFO: `/v3/users/d1828412cb04465dbfa15b26582aa6de` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:27.060570 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d798ed89-c16f-4f81-ae38-f7ffdf7f1287 None admin] RBAC: Authorizing `identity:get_user(user_id=d1828412cb04465dbfa15b26582aa6de)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:27.062599 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d798ed89-c16f-4f81-ae38-f7ffdf7f1287 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:27.064238 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 59/122] 10.222.0.16 () {56 vars in 1177 bytes} [Wed Aug 24 09:07:26 2022] GET /identity/v3/users/d1828412cb04465dbfa15b26582aa6de => generated 247 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:07:27.074710 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d798ed89-c16f-4f81-ae38-f7ffdf7f1287 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:27.104978 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ea09c8-5bfa-47cf-8b4e-50d52c45d0bf None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:27.141756 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ef2f70-08b2-4772-85c8-98b92dd3f9a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:27.142382 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ef2f70-08b2-4772-85c8-98b92dd3f9a4 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:27.142665 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ef2f70-08b2-4772-85c8-98b92dd3f9a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:27.142989 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ef2f70-08b2-4772-85c8-98b92dd3f9a4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:27.158504 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:27.160317 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:27.188216 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d798ed89-c16f-4f81-ae38-f7ffdf7f1287 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:27.289406 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ef2f70-08b2-4772-85c8-98b92dd3f9a4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:27.291167 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ef2f70-08b2-4772-85c8-98b92dd3f9a4 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:27.329154 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5893b3-9b65-4d16-835c-44a8a18bc73f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:27.329154 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5893b3-9b65-4d16-835c-44a8a18bc73f None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:27.329405 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5893b3-9b65-4d16-835c-44a8a18bc73f None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:27.329564 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5893b3-9b65-4d16-835c-44a8a18bc73f None admin] PATH_INFO: `/v3/projects/7d6be68cf8b046d5b03fba3c5a367c11` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:27.336550 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5893b3-9b65-4d16-835c-44a8a18bc73f None admin] RBAC: Authorizing `identity:get_project(project_id=7d6be68cf8b046d5b03fba3c5a367c11)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:27.337853 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5893b3-9b65-4d16-835c-44a8a18bc73f None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:27.339573 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 60/123] 10.222.0.16 () {56 vars in 1186 bytes} [Wed Aug 24 09:07:27 2022] GET /identity/v3/projects/7d6be68cf8b046d5b03fba3c5a367c11 => generated 298 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:07:27.340372 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5893b3-9b65-4d16-835c-44a8a18bc73f None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:27.401783 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 64/124] 10.222.0.16 () {58 vars in 1282 bytes} [Wed Aug 24 09:07:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 09:07:27.403014 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ef2f70-08b2-4772-85c8-98b92dd3f9a4 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:27.453260 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:27.512105 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ef2f70-08b2-4772-85c8-98b92dd3f9a4 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:27.517376 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5893b3-9b65-4d16-835c-44a8a18bc73f None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:27.558670 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292a38e9-fea0-448a-a8b1-f821653b2063 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:27.559348 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292a38e9-fea0-448a-a8b1-f821653b2063 None admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:27.559717 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292a38e9-fea0-448a-a8b1-f821653b2063 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:27.559998 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292a38e9-fea0-448a-a8b1-f821653b2063 None admin] PATH_INFO: `/v3/projects/7d6be68cf8b046d5b03fba3c5a367c11/users/d1828412cb04465dbfa15b26582aa6de/roles/15f634d4a2d247ddab90c458e5e1c141` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:27.562538 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292a38e9-fea0-448a-a8b1-f821653b2063 None admin] RBAC: Authorizing `identity:create_grant(project_id=7d6be68cf8b046d5b03fba3c5a367c11, user_id=d1828412cb04465dbfa15b26582aa6de, role_id=15f634d4a2d247ddab90c458e5e1c141)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:27.563230 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180f3a5e-8774-4b19-8a51-75c11272bca2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:27.564072 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292a38e9-fea0-448a-a8b1-f821653b2063 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:27.564329 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180f3a5e-8774-4b19-8a51-75c11272bca2 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:27.564799 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180f3a5e-8774-4b19-8a51-75c11272bca2 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:27.565277 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180f3a5e-8774-4b19-8a51-75c11272bca2 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:27.580240 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180f3a5e-8774-4b19-8a51-75c11272bca2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:07:27.585271 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-180f3a5e-8774-4b19-8a51-75c11272bca2 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 09:07:27.585271 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:07:27.585271 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:07:27.585271 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:07:27.585271 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:07:27.585271 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:07:27.585271 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:07:27.585271 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:07:27.585271 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:07:27.585271 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:07:27.585271 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:07:27.585271 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:07:27.585271 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:07:27.585271 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:07:27.585271 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:07:27.585271 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:07:27.586382 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:07:27.586382 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:07:27.586382 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:07:27.586382 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:07:27.586382 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:07:27.586382 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:07:27.586382 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:07:27.586382 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:07:27.586382 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:07:27.586382 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:07:27.586382 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:07:27.586382 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 24 09:07:27.586382 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:07:27.586382 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:07:27.586382 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:07:27.586382 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:07:27.587372 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:07:27.587372 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:07:27.587372 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:07:27.587372 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:07:27.587372 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:07:27.587372 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:07:27.587372 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:07:27.587372 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:07:27.587372 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:07:27.587372 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:07:27.587372 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 09:07:27.587372 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 24 09:07:27.589482 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 61/125] 10.222.0.16 () {56 vars in 1114 bytes} [Wed Aug 24 09:07:27 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:07:27.590432 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180f3a5e-8774-4b19-8a51-75c11272bca2 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:27.594346 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:27.657173 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 65/126] 10.222.0.16 () {58 vars in 1439 bytes} [Wed Aug 24 09:07:27 2022] PUT /identity/v3/projects/7d6be68cf8b046d5b03fba3c5a367c11/users/d1828412cb04465dbfa15b26582aa6de/roles/15f634d4a2d247ddab90c458e5e1c141 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:07:27.658446 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292a38e9-fea0-448a-a8b1-f821653b2063 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:27.755048 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180f3a5e-8774-4b19-8a51-75c11272bca2 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:27.809882 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a97add-8e4a-4e78-8c0c-bc7f90f85201 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:27.810534 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a97add-8e4a-4e78-8c0c-bc7f90f85201 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:27.811399 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a97add-8e4a-4e78-8c0c-bc7f90f85201 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:27.811693 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a97add-8e4a-4e78-8c0c-bc7f90f85201 None admin] PATH_INFO: `/v3/projects/7d6be68cf8b046d5b03fba3c5a367c11` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:27.812857 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a97add-8e4a-4e78-8c0c-bc7f90f85201 None admin] RBAC: Authorizing `identity:get_project(project_id=7d6be68cf8b046d5b03fba3c5a367c11)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:27.814342 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a97add-8e4a-4e78-8c0c-bc7f90f85201 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:27.817296 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 62/127] 10.222.0.16 () {56 vars in 1186 bytes} [Wed Aug 24 09:07:27 2022] GET /identity/v3/projects/7d6be68cf8b046d5b03fba3c5a367c11 => generated 298 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:07:27.817296 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a97add-8e4a-4e78-8c0c-bc7f90f85201 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:27.847825 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292a38e9-fea0-448a-a8b1-f821653b2063 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:27.850649 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:27.887902 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0af397-d3eb-472b-a84f-a287d68751eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:27.888756 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0af397-d3eb-472b-a84f-a287d68751eb None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:27.889232 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0af397-d3eb-472b-a84f-a287d68751eb None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:27.889602 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0af397-d3eb-472b-a84f-a287d68751eb None admin] PATH_INFO: `/v3/roles/member` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:27.903130 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:27.912023 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0af397-d3eb-472b-a84f-a287d68751eb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:07:27.921020 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-fe0af397-d3eb-472b-a84f-a287d68751eb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 09:07:27.921020 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:07:27.921020 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:07:27.921020 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:07:27.921020 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:07:27.921020 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:07:27.921020 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:07:27.921020 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:07:27.921020 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:07:27.921020 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:07:27.921020 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:07:27.921020 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:07:27.921020 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:07:27.921020 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:07:27.921020 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:07:27.921020 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:07:27.923858 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:07:27.923858 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:07:27.923858 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:07:27.923858 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:07:27.923858 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:07:27.923858 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:07:27.923858 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:07:27.923858 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:07:27.923858 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:07:27.923858 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:07:27.923858 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:07:27.923858 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Aug 24 09:07:27.923858 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:07:27.923858 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:07:27.923858 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:07:27.923858 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:07:27.924655 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:07:27.924655 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:07:27.924655 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:07:27.924655 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:07:27.924655 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:07:27.924655 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:07:27.924655 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:07:27.924655 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:07:27.924655 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:07:27.924655 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:07:27.924655 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 09:07:27.924655 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Aug 24 09:07:27.925778 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 66/128] 10.222.0.16 () {56 vars in 1099 bytes} [Wed Aug 24 09:07:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:07:27.925778 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0af397-d3eb-472b-a84f-a287d68751eb None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:27.955541 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a97add-8e4a-4e78-8c0c-bc7f90f85201 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:28.027341 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b161f3f7-4b07-47d1-88c0-7f616def69c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:28.027983 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b161f3f7-4b07-47d1-88c0-7f616def69c0 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:28.028274 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b161f3f7-4b07-47d1-88c0-7f616def69c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:28.028557 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b161f3f7-4b07-47d1-88c0-7f616def69c0 None admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:28.029236 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b161f3f7-4b07-47d1-88c0-7f616def69c0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:07:28.030074 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b161f3f7-4b07-47d1-88c0-7f616def69c0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:28.032794 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b161f3f7-4b07-47d1-88c0-7f616def69c0 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:28.050116 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 63/129] 10.222.0.16 () {56 vars in 1111 bytes} [Wed Aug 24 09:07:27 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:07:28.050807 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b161f3f7-4b07-47d1-88c0-7f616def69c0 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:28.133612 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0af397-d3eb-472b-a84f-a287d68751eb None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:28.182794 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b161f3f7-4b07-47d1-88c0-7f616def69c0 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:28.221678 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:28.228320 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea036b8-22ab-47af-9ae7-821ccbc50c02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:28.229062 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea036b8-22ab-47af-9ae7-821ccbc50c02 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:28.229359 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea036b8-22ab-47af-9ae7-821ccbc50c02 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:28.229646 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea036b8-22ab-47af-9ae7-821ccbc50c02 None admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:28.230346 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea036b8-22ab-47af-9ae7-821ccbc50c02 None admin] RBAC: Adding query filter params (name=member) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:07:28.230658 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea036b8-22ab-47af-9ae7-821ccbc50c02 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:28.232236 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea036b8-22ab-47af-9ae7-821ccbc50c02 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:28.240954 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac460e04-5326-4132-ba04-99baad99f973 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:28.241587 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac460e04-5326-4132-ba04-99baad99f973 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:28.241871 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac460e04-5326-4132-ba04-99baad99f973 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:28.242149 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac460e04-5326-4132-ba04-99baad99f973 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:28.242807 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 64/130] 10.222.0.16 () {56 vars in 1101 bytes} [Wed Aug 24 09:07:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:07:28.242909 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac460e04-5326-4132-ba04-99baad99f973 None admin] RBAC: Adding query filter params (role.id=5a55760968cc43ef86318e5bec24f4ef, scope.project.id=7d6be68cf8b046d5b03fba3c5a367c11, user.id=693fc8b7114049948366081c3e04537c) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:07:28.243198 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac460e04-5326-4132-ba04-99baad99f973 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:28.243406 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea036b8-22ab-47af-9ae7-821ccbc50c02 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:28.244694 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac460e04-5326-4132-ba04-99baad99f973 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:28.269397 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 67/131] 10.222.0.16 () {56 vars in 1374 bytes} [Wed Aug 24 09:07:27 2022] GET /identity/v3/role_assignments?user.id=693fc8b7114049948366081c3e04537c&scope.project.id=7d6be68cf8b046d5b03fba3c5a367c11&role.id=5a55760968cc43ef86318e5bec24f4ef => generated 259 bytes in 344 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:07:28.270307 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac460e04-5326-4132-ba04-99baad99f973 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:28.368732 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:28.435662 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea036b8-22ab-47af-9ae7-821ccbc50c02 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:28.493842 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36ad2c4-2baf-4497-aec0-c0bc34015b35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:28.494669 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36ad2c4-2baf-4497-aec0-c0bc34015b35 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:28.494965 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36ad2c4-2baf-4497-aec0-c0bc34015b35 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:28.495244 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36ad2c4-2baf-4497-aec0-c0bc34015b35 None admin] PATH_INFO: `/v3/users/1925e7a247b64a06b5abece3c5687b12` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:28.500308 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36ad2c4-2baf-4497-aec0-c0bc34015b35 None admin] RBAC: Authorizing `identity:get_user(user_id=1925e7a247b64a06b5abece3c5687b12)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:28.501777 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36ad2c4-2baf-4497-aec0-c0bc34015b35 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:28.503405 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 65/132] 10.222.0.16 () {56 vars in 1177 bytes} [Wed Aug 24 09:07:28 2022] GET /identity/v3/users/1925e7a247b64a06b5abece3c5687b12 => generated 275 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:07:28.512673 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36ad2c4-2baf-4497-aec0-c0bc34015b35 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:28.538405 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac460e04-5326-4132-ba04-99baad99f973 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:28.588241 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:28.633572 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976718dd-e4a7-410d-a88b-fc719a33e3fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:28.634233 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976718dd-e4a7-410d-a88b-fc719a33e3fa None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:28.634504 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976718dd-e4a7-410d-a88b-fc719a33e3fa None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:28.634782 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976718dd-e4a7-410d-a88b-fc719a33e3fa None admin] PATH_INFO: `/v3/users/1925e7a247b64a06b5abece3c5687b12` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:28.636915 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976718dd-e4a7-410d-a88b-fc719a33e3fa None admin] RBAC: Authorizing `identity:get_user(user_id=1925e7a247b64a06b5abece3c5687b12)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:28.638389 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976718dd-e4a7-410d-a88b-fc719a33e3fa None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:28.640211 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 68/133] 10.222.0.16 () {56 vars in 1177 bytes} [Wed Aug 24 09:07:28 2022] GET /identity/v3/users/1925e7a247b64a06b5abece3c5687b12 => generated 275 bytes in 370 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:07:28.641180 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba13c9f4-9687-447b-b4f1-f3987125ef5e None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:28.641470 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba13c9f4-9687-447b-b4f1-f3987125ef5e None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:28.641778 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba13c9f4-9687-447b-b4f1-f3987125ef5e None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:28.643196 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 69/134] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:07:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:07:28.650287 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba13c9f4-9687-447b-b4f1-f3987125ef5e None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:28.666785 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36ad2c4-2baf-4497-aec0-c0bc34015b35 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:28.728187 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a68f93-f737-4de2-a4db-300209452656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:28.728758 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a68f93-f737-4de2-a4db-300209452656 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:28.729047 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a68f93-f737-4de2-a4db-300209452656 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:28.729325 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a68f93-f737-4de2-a4db-300209452656 None admin] PATH_INFO: `/v3/projects/7d6be68cf8b046d5b03fba3c5a367c11` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:28.731136 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a68f93-f737-4de2-a4db-300209452656 None admin] RBAC: Authorizing `identity:get_project(project_id=7d6be68cf8b046d5b03fba3c5a367c11)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:28.733206 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a68f93-f737-4de2-a4db-300209452656 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:28.735001 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 66/135] 10.222.0.16 () {56 vars in 1186 bytes} [Wed Aug 24 09:07:28 2022] GET /identity/v3/projects/7d6be68cf8b046d5b03fba3c5a367c11 => generated 298 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:07:28.740274 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e9df39-5ce2-455d-b43e-2d203d97241f None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:28.740575 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e9df39-5ce2-455d-b43e-2d203d97241f None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:28.740856 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e9df39-5ce2-455d-b43e-2d203d97241f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:28.773766 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-24e9df39-5ce2-455d-b43e-2d203d97241f None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:07:28.774975 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-24e9df39-5ce2-455d-b43e-2d203d97241f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:07:28.827815 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:28.838323 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba13c9f4-9687-447b-b4f1-f3987125ef5e None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:28.882820 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648d6298-8584-44d2-9e2d-68c86c0c094a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:28.883463 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648d6298-8584-44d2-9e2d-68c86c0c094a None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:28.883769 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648d6298-8584-44d2-9e2d-68c86c0c094a None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:28.884045 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648d6298-8584-44d2-9e2d-68c86c0c094a None admin] PATH_INFO: `/v3/projects/d6829a2bfe5a433dbc0b7b3dc0283fdf` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:28.885316 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648d6298-8584-44d2-9e2d-68c86c0c094a None admin] RBAC: Authorizing `identity:get_project(project_id=d6829a2bfe5a433dbc0b7b3dc0283fdf)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:28.890523 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648d6298-8584-44d2-9e2d-68c86c0c094a None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:28.892251 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 70/136] 10.222.0.16 () {56 vars in 1186 bytes} [Wed Aug 24 09:07:28 2022] GET /identity/v3/projects/d6829a2bfe5a433dbc0b7b3dc0283fdf => generated 312 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:07:28.894141 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648d6298-8584-44d2-9e2d-68c86c0c094a None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:28.917226 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 67/137] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:07:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:07:28.918152 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e9df39-5ce2-455d-b43e-2d203d97241f None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:29.053433 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648d6298-8584-44d2-9e2d-68c86c0c094a None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:29.061052 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:29.098576 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed97103-e86c-4f6e-8c1b-e352b6017801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:29.099296 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed97103-e86c-4f6e-8c1b-e352b6017801 None admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:29.099598 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed97103-e86c-4f6e-8c1b-e352b6017801 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:29.099872 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed97103-e86c-4f6e-8c1b-e352b6017801 None admin] PATH_INFO: `/v3/projects/7d6be68cf8b046d5b03fba3c5a367c11/users/1925e7a247b64a06b5abece3c5687b12/roles/ed6466d17fb7403c8491bb7f1ef95fc1` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:29.103357 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed97103-e86c-4f6e-8c1b-e352b6017801 None admin] RBAC: Authorizing `identity:create_grant(project_id=7d6be68cf8b046d5b03fba3c5a367c11, user_id=1925e7a247b64a06b5abece3c5687b12, role_id=ed6466d17fb7403c8491bb7f1ef95fc1)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:29.104866 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed97103-e86c-4f6e-8c1b-e352b6017801 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:29.156026 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e9df39-5ce2-455d-b43e-2d203d97241f None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:29.180057 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 71/138] 10.222.0.16 () {58 vars in 1439 bytes} [Wed Aug 24 09:07:28 2022] PUT /identity/v3/projects/7d6be68cf8b046d5b03fba3c5a367c11/users/1925e7a247b64a06b5abece3c5687b12/roles/ed6466d17fb7403c8491bb7f1ef95fc1 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:07:29.225509 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0b6e0d-c310-41f1-a9e3-d01b3c5da103 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:29.226206 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0b6e0d-c310-41f1-a9e3-d01b3c5da103 None admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:29.226492 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0b6e0d-c310-41f1-a9e3-d01b3c5da103 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:29.226771 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0b6e0d-c310-41f1-a9e3-d01b3c5da103 None admin] PATH_INFO: `/v3/projects/d6829a2bfe5a433dbc0b7b3dc0283fdf/users/1925e7a247b64a06b5abece3c5687b12/roles/136cdf78e00a426997dd4e37faff489b` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:29.228771 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0b6e0d-c310-41f1-a9e3-d01b3c5da103 None admin] RBAC: Authorizing `identity:create_grant(project_id=d6829a2bfe5a433dbc0b7b3dc0283fdf, user_id=1925e7a247b64a06b5abece3c5687b12, role_id=136cdf78e00a426997dd4e37faff489b)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:29.230231 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0b6e0d-c310-41f1-a9e3-d01b3c5da103 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:29.269007 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed97103-e86c-4f6e-8c1b-e352b6017801 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:29.313367 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:29.333940 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 68/139] 10.222.0.16 () {58 vars in 1439 bytes} [Wed Aug 24 09:07:28 2022] PUT /identity/v3/projects/d6829a2bfe5a433dbc0b7b3dc0283fdf/users/1925e7a247b64a06b5abece3c5687b12/roles/136cdf78e00a426997dd4e37faff489b => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:07:29.373503 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ef7898-a2f2-4fcd-91af-a35763c83244 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:29.373819 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ef7898-a2f2-4fcd-91af-a35763c83244 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:29.374082 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ef7898-a2f2-4fcd-91af-a35763c83244 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:29.375775 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 69/140] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:07:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:07:29.381565 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634dd2a7-b21e-407d-ab1c-37deb9122a47 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:29.381840 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634dd2a7-b21e-407d-ab1c-37deb9122a47 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:29.382117 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634dd2a7-b21e-407d-ab1c-37deb9122a47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:29.424874 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed97103-e86c-4f6e-8c1b-e352b6017801 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:29.428730 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-634dd2a7-b21e-407d-ab1c-37deb9122a47 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:07:29.429963 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-634dd2a7-b21e-407d-ab1c-37deb9122a47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:07:29.495407 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8785c8c5-c02c-4764-a1ff-551ea7e7eb30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:29.496267 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8785c8c5-c02c-4764-a1ff-551ea7e7eb30 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:29.496557 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8785c8c5-c02c-4764-a1ff-551ea7e7eb30 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:29.496833 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8785c8c5-c02c-4764-a1ff-551ea7e7eb30 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:29.515875 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8785c8c5-c02c-4764-a1ff-551ea7e7eb30 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:07:29.530652 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-8785c8c5-c02c-4764-a1ff-551ea7e7eb30 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 09:07:29.530652 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:07:29.530652 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:07:29.530652 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:07:29.530652 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:07:29.530652 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:07:29.530652 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:07:29.530652 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:07:29.530652 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:07:29.530652 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:07:29.530652 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:07:29.530652 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:07:29.530652 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:07:29.530652 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:07:29.530652 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:07:29.530652 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:07:29.531623 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:07:29.531623 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:07:29.531623 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:07:29.531623 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:07:29.531623 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:07:29.531623 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:07:29.531623 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:07:29.531623 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:07:29.531623 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:07:29.531623 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:07:29.531623 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:07:29.531623 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Aug 24 09:07:29.531623 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:07:29.531623 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:07:29.531623 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:07:29.531623 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:07:29.533072 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:07:29.533072 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:07:29.533072 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:07:29.533072 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:07:29.533072 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:07:29.533072 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:07:29.533072 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:07:29.533072 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:07:29.533072 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:07:29.533072 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:07:29.533072 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 09:07:29.533072 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Aug 24 09:07:29.533072 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 72/141] 10.222.0.16 () {56 vars in 1099 bytes} [Wed Aug 24 09:07:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 09:07:29.539853 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8785c8c5-c02c-4764-a1ff-551ea7e7eb30 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:29.581618 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 70/142] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:07:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 09:07:29.651538 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:29.673799 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8785c8c5-c02c-4764-a1ff-551ea7e7eb30 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:29.726012 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f418ad-260d-492c-a178-16481b8904dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:29.726641 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f418ad-260d-492c-a178-16481b8904dd None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:29.726929 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f418ad-260d-492c-a178-16481b8904dd None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:29.727207 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f418ad-260d-492c-a178-16481b8904dd None admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:29.727919 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f418ad-260d-492c-a178-16481b8904dd None admin] RBAC: Adding query filter params (name=member) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:07:29.728233 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f418ad-260d-492c-a178-16481b8904dd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:29.729621 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f418ad-260d-492c-a178-16481b8904dd None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:29.744879 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 73/143] 10.222.0.16 () {56 vars in 1101 bytes} [Wed Aug 24 09:07:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:07:29.750411 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634dd2a7-b21e-407d-ab1c-37deb9122a47 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:29.836799 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:29.907175 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634dd2a7-b21e-407d-ab1c-37deb9122a47 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:29.943905 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9f5e2a-437d-4806-83c8-a248e566f776 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:29.944596 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9f5e2a-437d-4806-83c8-a248e566f776 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:29.944883 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9f5e2a-437d-4806-83c8-a248e566f776 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:29.945156 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9f5e2a-437d-4806-83c8-a248e566f776 None admin] PATH_INFO: `/v3/users/1925e7a247b64a06b5abece3c5687b12` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:29.946291 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9f5e2a-437d-4806-83c8-a248e566f776 None admin] RBAC: Authorizing `identity:get_user(user_id=1925e7a247b64a06b5abece3c5687b12)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:29.947876 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9f5e2a-437d-4806-83c8-a248e566f776 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:29.949456 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 71/144] 10.222.0.16 () {56 vars in 1177 bytes} [Wed Aug 24 09:07:29 2022] GET /identity/v3/users/1925e7a247b64a06b5abece3c5687b12 => generated 275 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:07:29.955425 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f418ad-260d-492c-a178-16481b8904dd None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:30.091385 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87aaa72c-f1f2-457e-a0dd-1767582af65d None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:30.092065 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87aaa72c-f1f2-457e-a0dd-1767582af65d None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:30.092566 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87aaa72c-f1f2-457e-a0dd-1767582af65d None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:30.094110 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 72/145] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:07:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:07:30.098826 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f418ad-260d-492c-a178-16481b8904dd None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:30.111769 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca5afb7-bdd1-4267-b90c-6ac975611fdc None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:30.112357 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca5afb7-bdd1-4267-b90c-6ac975611fdc None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:30.112700 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca5afb7-bdd1-4267-b90c-6ac975611fdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:30.140794 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75eff31f-669b-4b98-9558-034e62d84c2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:30.141703 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75eff31f-669b-4b98-9558-034e62d84c2f None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:30.142151 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75eff31f-669b-4b98-9558-034e62d84c2f None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:30.144996 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75eff31f-669b-4b98-9558-034e62d84c2f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:30.152634 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-4ca5afb7-bdd1-4267-b90c-6ac975611fdc None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:07:30.153849 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-4ca5afb7-bdd1-4267-b90c-6ac975611fdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:07:30.161424 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:30.271137 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75eff31f-669b-4b98-9558-034e62d84c2f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:30.274760 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75eff31f-669b-4b98-9558-034e62d84c2f None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:30.292408 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 73/146] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:07:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:07:30.293987 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca5afb7-bdd1-4267-b90c-6ac975611fdc None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:30.411641 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:30.475012 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 74/147] 10.222.0.16 () {58 vars in 1282 bytes} [Wed Aug 24 09:07:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 520 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 09:07:30.482150 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75eff31f-669b-4b98-9558-034e62d84c2f None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:30.508998 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca5afb7-bdd1-4267-b90c-6ac975611fdc None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:30.563619 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:30.584482 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fdc13a-fcdf-4e15-9285-66da8ea57bcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:30.585315 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fdc13a-fcdf-4e15-9285-66da8ea57bcf None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:30.585603 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fdc13a-fcdf-4e15-9285-66da8ea57bcf None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:30.585875 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fdc13a-fcdf-4e15-9285-66da8ea57bcf None admin] PATH_INFO: `/v3/domains/default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:30.588043 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fdc13a-fcdf-4e15-9285-66da8ea57bcf None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:30.589490 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fdc13a-fcdf-4e15-9285-66da8ea57bcf None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:30.594017 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 74/148] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 24 09:07:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:07:30.599314 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fdc13a-fcdf-4e15-9285-66da8ea57bcf None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:30.647828 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75eff31f-669b-4b98-9558-034e62d84c2f None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:30.696400 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e5c1df-2414-4a3a-9b7c-c0b10f9fe80a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:30.696996 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e5c1df-2414-4a3a-9b7c-c0b10f9fe80a None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:30.697237 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e5c1df-2414-4a3a-9b7c-c0b10f9fe80a None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:30.697518 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e5c1df-2414-4a3a-9b7c-c0b10f9fe80a None admin] PATH_INFO: `/v3/projects/7d6be68cf8b046d5b03fba3c5a367c11` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:30.698697 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e5c1df-2414-4a3a-9b7c-c0b10f9fe80a None admin] RBAC: Authorizing `identity:get_project(project_id=7d6be68cf8b046d5b03fba3c5a367c11)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:30.700197 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e5c1df-2414-4a3a-9b7c-c0b10f9fe80a None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:30.702851 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 75/149] 10.222.0.16 () {56 vars in 1186 bytes} [Wed Aug 24 09:07:30 2022] GET /identity/v3/projects/7d6be68cf8b046d5b03fba3c5a367c11 => generated 298 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:07:30.708509 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e5c1df-2414-4a3a-9b7c-c0b10f9fe80a None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:30.823868 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fdc13a-fcdf-4e15-9285-66da8ea57bcf None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:30.851957 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:30.871169 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed5a5c4-ae0a-43d7-b3cc-d6d7d1ffae88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:30.871817 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed5a5c4-ae0a-43d7-b3cc-d6d7d1ffae88 None admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:30.872102 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed5a5c4-ae0a-43d7-b3cc-d6d7d1ffae88 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:30.872378 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed5a5c4-ae0a-43d7-b3cc-d6d7d1ffae88 None admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:30.873782 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed5a5c4-ae0a-43d7-b3cc-d6d7d1ffae88 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:30.875258 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed5a5c4-ae0a-43d7-b3cc-d6d7d1ffae88 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:30.894664 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e5c1df-2414-4a3a-9b7c-c0b10f9fe80a None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:30.916625 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:30.939519 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5476fbc5-0dc5-444e-8784-6d1f335710f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:30.940255 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5476fbc5-0dc5-444e-8784-6d1f335710f1 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:30.940595 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5476fbc5-0dc5-444e-8784-6d1f335710f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:30.940878 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5476fbc5-0dc5-444e-8784-6d1f335710f1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:30.941725 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5476fbc5-0dc5-444e-8784-6d1f335710f1 None admin] RBAC: Adding query filter params (role.id=136cdf78e00a426997dd4e37faff489b, scope.project.id=7d6be68cf8b046d5b03fba3c5a367c11, user.id=1925e7a247b64a06b5abece3c5687b12) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:07:30.942043 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5476fbc5-0dc5-444e-8784-6d1f335710f1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:30.943613 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5476fbc5-0dc5-444e-8784-6d1f335710f1 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:30.950472 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 76/150] 10.222.0.16 () {56 vars in 1374 bytes} [Wed Aug 24 09:07:30 2022] GET /identity/v3/role_assignments?user.id=1925e7a247b64a06b5abece3c5687b12&scope.project.id=7d6be68cf8b046d5b03fba3c5a367c11&role.id=136cdf78e00a426997dd4e37faff489b => generated 608 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:07:30.951492 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5476fbc5-0dc5-444e-8784-6d1f335710f1 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:31.013791 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 75/151] 10.222.0.16 () {60 vars in 1132 bytes} [Wed Aug 24 09:07:30 2022] POST /identity/v3/users => generated 283 bytes in 417 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 09:07:31.031258 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a7dff1-dfe7-4feb-8366-7adf6fed9daf None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:31.031555 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a7dff1-dfe7-4feb-8366-7adf6fed9daf None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:31.031841 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a7dff1-dfe7-4feb-8366-7adf6fed9daf None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:31.033342 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 76/152] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:07:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 09:07:31.045851 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8f8a15-84eb-413e-b0b0-a0789dc2d39e None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:31.046466 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8f8a15-84eb-413e-b0b0-a0789dc2d39e None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:31.046839 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8f8a15-84eb-413e-b0b0-a0789dc2d39e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:31.093271 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-9b8f8a15-84eb-413e-b0b0-a0789dc2d39e None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:07:31.094496 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-9b8f8a15-84eb-413e-b0b0-a0789dc2d39e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:07:31.100993 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5476fbc5-0dc5-444e-8784-6d1f335710f1 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:31.141852 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c5fe22-2a95-448a-97cf-15e4de3ea089 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:31.142838 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c5fe22-2a95-448a-97cf-15e4de3ea089 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:31.143363 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c5fe22-2a95-448a-97cf-15e4de3ea089 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:31.144630 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c5fe22-2a95-448a-97cf-15e4de3ea089 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:31.173041 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c5fe22-2a95-448a-97cf-15e4de3ea089 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:07:31.179892 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-15c5fe22-2a95-448a-97cf-15e4de3ea089 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:07:31.179892 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:07:31.179892 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:07:31.179892 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:07:31.179892 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:07:31.179892 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:07:31.179892 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:07:31.179892 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:07:31.179892 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:07:31.179892 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:07:31.179892 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:07:31.179892 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:07:31.179892 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:07:31.179892 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:07:31.179892 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:07:31.179892 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:07:31.191446 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:07:31.191446 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:07:31.191446 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:07:31.191446 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:07:31.191446 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:07:31.191446 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:07:31.191446 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:07:31.191446 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:07:31.191446 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:07:31.191446 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:07:31.191446 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:07:31.191446 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Aug 24 09:07:31.191446 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:07:31.191446 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:07:31.191446 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:07:31.191446 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:07:31.192452 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:07:31.192452 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:07:31.192452 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:07:31.192452 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:07:31.192452 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:07:31.192452 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:07:31.192452 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:07:31.192452 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:07:31.192452 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:07:31.192452 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:07:31.192452 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:07:31.192452 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Aug 24 09:07:31.192452 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 77/153] 10.222.0.16 () {56 vars in 1096 bytes} [Wed Aug 24 09:07:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:07:31.192452 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c5fe22-2a95-448a-97cf-15e4de3ea089 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:31.283959 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:31.322695 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 77/154] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:07:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:07:31.373403 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c5fe22-2a95-448a-97cf-15e4de3ea089 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:31.412953 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8c2b81-e5b5-4ba1-b446-6332c9a17689 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:31.413938 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8c2b81-e5b5-4ba1-b446-6332c9a17689 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:31.414421 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8c2b81-e5b5-4ba1-b446-6332c9a17689 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:31.414892 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8c2b81-e5b5-4ba1-b446-6332c9a17689 None admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:31.415824 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8c2b81-e5b5-4ba1-b446-6332c9a17689 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:07:31.416332 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8c2b81-e5b5-4ba1-b446-6332c9a17689 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:31.417928 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8c2b81-e5b5-4ba1-b446-6332c9a17689 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:31.428275 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 78/155] 10.222.0.16 () {56 vars in 1099 bytes} [Wed Aug 24 09:07:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:07:31.436244 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8f8a15-84eb-413e-b0b0-a0789dc2d39e None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:31.558713 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8f8a15-84eb-413e-b0b0-a0789dc2d39e None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:31.561264 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:31.596367 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6211f793-6c77-4c04-becd-2cedfd13505a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:31.600357 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6211f793-6c77-4c04-becd-2cedfd13505a None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:31.600858 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6211f793-6c77-4c04-becd-2cedfd13505a None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:31.601346 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6211f793-6c77-4c04-becd-2cedfd13505a None admin] PATH_INFO: `/v3/users/d1828412cb04465dbfa15b26582aa6de` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:31.602564 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6211f793-6c77-4c04-becd-2cedfd13505a None admin] RBAC: Authorizing `identity:get_user(user_id=d1828412cb04465dbfa15b26582aa6de)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:31.604060 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6211f793-6c77-4c04-becd-2cedfd13505a None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:31.608624 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 78/156] 10.222.0.16 () {56 vars in 1177 bytes} [Wed Aug 24 09:07:31 2022] GET /identity/v3/users/d1828412cb04465dbfa15b26582aa6de => generated 247 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:07:31.612547 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8c2b81-e5b5-4ba1-b446-6332c9a17689 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:31.706252 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6211f793-6c77-4c04-becd-2cedfd13505a None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:31.746108 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:31.750487 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8c2b81-e5b5-4ba1-b446-6332c9a17689 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:31.794169 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a503b6a-2b5c-4a2d-8df8-ce1e87433af7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:31.795951 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a503b6a-2b5c-4a2d-8df8-ce1e87433af7 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:31.796381 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a503b6a-2b5c-4a2d-8df8-ce1e87433af7 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:31.796821 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a503b6a-2b5c-4a2d-8df8-ce1e87433af7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:31.835065 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6211f793-6c77-4c04-becd-2cedfd13505a None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:31.874330 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e150bec8-bfa6-4085-a424-688708d2db33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:31.875299 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e150bec8-bfa6-4085-a424-688708d2db33 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:31.875777 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e150bec8-bfa6-4085-a424-688708d2db33 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:31.876233 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e150bec8-bfa6-4085-a424-688708d2db33 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:31.880310 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a503b6a-2b5c-4a2d-8df8-ce1e87433af7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:31.881040 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a503b6a-2b5c-4a2d-8df8-ce1e87433af7 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:31.896544 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e150bec8-bfa6-4085-a424-688708d2db33 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:07:31.901376 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-e150bec8-bfa6-4085-a424-688708d2db33 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 09:07:31.901376 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:07:31.901376 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:07:31.901376 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:07:31.901376 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:07:31.901376 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:07:31.901376 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:07:31.901376 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:07:31.901376 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:07:31.901376 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:07:31.901376 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:07:31.901376 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:07:31.901376 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:07:31.901376 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:07:31.901376 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:07:31.901376 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:07:31.902487 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:07:31.902487 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:07:31.902487 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:07:31.902487 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:07:31.902487 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:07:31.902487 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:07:31.902487 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:07:31.902487 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:07:31.902487 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:07:31.902487 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:07:31.902487 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:07:31.902487 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 24 09:07:31.902487 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:07:31.902487 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:07:31.902487 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:07:31.902487 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:07:31.903462 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:07:31.903462 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:07:31.903462 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:07:31.903462 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:07:31.903462 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:07:31.903462 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:07:31.903462 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:07:31.903462 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:07:31.903462 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:07:31.903462 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:07:31.903462 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 09:07:31.903462 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 24 09:07:31.903462 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 79/157] 10.222.0.16 () {56 vars in 1099 bytes} [Wed Aug 24 09:07:31 2022] GET /identity/v3/roles/reader => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 09:07:31.908425 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f658cb40-6121-47d1-8add-85a9d2dd445a None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:31.908425 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f658cb40-6121-47d1-8add-85a9d2dd445a None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:31.908425 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f658cb40-6121-47d1-8add-85a9d2dd445a None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:31.908425 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 80/158] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:07:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:07:31.912142 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfae0f9-a20c-4a56-84fe-2f1bd39a7182 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:31.912425 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfae0f9-a20c-4a56-84fe-2f1bd39a7182 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:31.912701 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfae0f9-a20c-4a56-84fe-2f1bd39a7182 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:31.931526 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-3bfae0f9-a20c-4a56-84fe-2f1bd39a7182 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:07:31.932743 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-3bfae0f9-a20c-4a56-84fe-2f1bd39a7182 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:07:31.957905 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:32.007891 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:32.012046 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 79/159] 10.222.0.16 () {58 vars in 1282 bytes} [Wed Aug 24 09:07:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 09:07:32.015622 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 81/160] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:07:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 09:07:32.016473 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a503b6a-2b5c-4a2d-8df8-ce1e87433af7 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:32.016875 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9896e423-684e-4cb5-bca6-cdd03c4deeaa None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:32.017161 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9896e423-684e-4cb5-bca6-cdd03c4deeaa None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:32.017437 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9896e423-684e-4cb5-bca6-cdd03c4deeaa None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:32.018744 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 82/161] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:07:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:07:32.021746 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9896e423-684e-4cb5-bca6-cdd03c4deeaa None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:32.136796 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9896e423-684e-4cb5-bca6-cdd03c4deeaa None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:32.137020 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a503b6a-2b5c-4a2d-8df8-ce1e87433af7 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:32.175596 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca36135-c1c1-48b5-ba3d-3ea64e106e79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:32.176264 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca36135-c1c1-48b5-ba3d-3ea64e106e79 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:32.176566 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca36135-c1c1-48b5-ba3d-3ea64e106e79 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:32.176803 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca36135-c1c1-48b5-ba3d-3ea64e106e79 None admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:32.177470 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca36135-c1c1-48b5-ba3d-3ea64e106e79 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:07:32.177772 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca36135-c1c1-48b5-ba3d-3ea64e106e79 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:32.178013 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435256a0-fed8-42ca-8dd0-037faf65e994 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:32.178860 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435256a0-fed8-42ca-8dd0-037faf65e994 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:32.179153 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca36135-c1c1-48b5-ba3d-3ea64e106e79 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:32.179548 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435256a0-fed8-42ca-8dd0-037faf65e994 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:32.180071 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435256a0-fed8-42ca-8dd0-037faf65e994 None admin] PATH_INFO: `/v3/projects/7d6be68cf8b046d5b03fba3c5a367c11` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:32.181637 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435256a0-fed8-42ca-8dd0-037faf65e994 None admin] RBAC: Authorizing `identity:get_project(project_id=7d6be68cf8b046d5b03fba3c5a367c11)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:32.183297 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435256a0-fed8-42ca-8dd0-037faf65e994 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:32.185360 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 83/162] 10.222.0.16 () {56 vars in 1186 bytes} [Wed Aug 24 09:07:32 2022] GET /identity/v3/projects/7d6be68cf8b046d5b03fba3c5a367c11 => generated 298 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:07:32.186771 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c17569d-2566-41e1-84d3-6fdc485fc523 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:32.187284 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c17569d-2566-41e1-84d3-6fdc485fc523 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:32.187785 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c17569d-2566-41e1-84d3-6fdc485fc523 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:32.192951 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 80/163] 10.222.0.16 () {56 vars in 1101 bytes} [Wed Aug 24 09:07:32 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:07:32.194572 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca36135-c1c1-48b5-ba3d-3ea64e106e79 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:32.209155 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-4c17569d-2566-41e1-84d3-6fdc485fc523 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:07:32.210843 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-4c17569d-2566-41e1-84d3-6fdc485fc523 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:07:32.238375 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:32.260016 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:32.289830 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 84/164] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:07:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:07:32.290998 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c17569d-2566-41e1-84d3-6fdc485fc523 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:32.307317 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca36135-c1c1-48b5-ba3d-3ea64e106e79 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:32.357392 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756f9215-9b86-420f-9051-fbb227c409d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:32.358046 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756f9215-9b86-420f-9051-fbb227c409d9 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:32.358328 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756f9215-9b86-420f-9051-fbb227c409d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:32.358600 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756f9215-9b86-420f-9051-fbb227c409d9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:32.359336 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756f9215-9b86-420f-9051-fbb227c409d9 None admin] RBAC: Adding query filter params (role.id=15f634d4a2d247ddab90c458e5e1c141, scope.project.id=7d6be68cf8b046d5b03fba3c5a367c11, user.id=d1828412cb04465dbfa15b26582aa6de) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:07:32.359674 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756f9215-9b86-420f-9051-fbb227c409d9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:32.361602 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756f9215-9b86-420f-9051-fbb227c409d9 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:32.368135 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 81/165] 10.222.0.16 () {56 vars in 1374 bytes} [Wed Aug 24 09:07:32 2022] GET /identity/v3/role_assignments?user.id=d1828412cb04465dbfa15b26582aa6de&scope.project.id=7d6be68cf8b046d5b03fba3c5a367c11&role.id=15f634d4a2d247ddab90c458e5e1c141 => generated 608 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:07:32.368953 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756f9215-9b86-420f-9051-fbb227c409d9 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:32.427607 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c17569d-2566-41e1-84d3-6fdc485fc523 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:32.504728 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70575958-9342-4915-8b0a-9f76695690c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:32.505389 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70575958-9342-4915-8b0a-9f76695690c7 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:32.505669 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70575958-9342-4915-8b0a-9f76695690c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:32.505936 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70575958-9342-4915-8b0a-9f76695690c7 None admin] PATH_INFO: `/v3/users/693fc8b7114049948366081c3e04537c` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:32.507297 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70575958-9342-4915-8b0a-9f76695690c7 None admin] RBAC: Authorizing `identity:get_user(user_id=693fc8b7114049948366081c3e04537c)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:32.509900 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70575958-9342-4915-8b0a-9f76695690c7 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:32.516502 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 85/166] 10.222.0.16 () {56 vars in 1177 bytes} [Wed Aug 24 09:07:32 2022] GET /identity/v3/users/693fc8b7114049948366081c3e04537c => generated 289 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:07:32.528394 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70575958-9342-4915-8b0a-9f76695690c7 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:32.572469 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756f9215-9b86-420f-9051-fbb227c409d9 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:32.577683 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:32.619797 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:32.631188 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b10f46-41f2-4879-9803-9e6950b3eec9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:32.632162 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b10f46-41f2-4879-9803-9e6950b3eec9 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:32.632902 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b10f46-41f2-4879-9803-9e6950b3eec9 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:32.633184 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b10f46-41f2-4879-9803-9e6950b3eec9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:32.647710 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b10f46-41f2-4879-9803-9e6950b3eec9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:07:32.654242 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-28b10f46-41f2-4879-9803-9e6950b3eec9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 09:07:32.654242 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:07:32.654242 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:07:32.654242 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:07:32.654242 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:07:32.654242 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:07:32.654242 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:07:32.654242 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:07:32.654242 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:07:32.654242 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:07:32.654242 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:07:32.654242 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:07:32.654242 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:07:32.654242 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:07:32.654242 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:07:32.654242 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:07:32.655916 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:07:32.655916 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:07:32.655916 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:07:32.655916 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:07:32.655916 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:07:32.655916 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:07:32.655916 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:07:32.655916 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:07:32.655916 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:07:32.655916 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:07:32.655916 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:07:32.655916 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Aug 24 09:07:32.655916 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:07:32.655916 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:07:32.655916 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:07:32.655916 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:07:32.657068 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:07:32.657068 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:07:32.657068 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:07:32.657068 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:07:32.657068 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:07:32.657068 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:07:32.657068 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:07:32.657068 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:07:32.657068 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:07:32.657068 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:07:32.657068 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 09:07:32.657068 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Aug 24 09:07:32.657068 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 82/167] 10.222.0.16 () {56 vars in 1099 bytes} [Wed Aug 24 09:07:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:07:32.663444 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b10f46-41f2-4879-9803-9e6950b3eec9 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:32.695130 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70575958-9342-4915-8b0a-9f76695690c7 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:32.738764 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b43cbee-0a4d-4cd6-857b-82f6904b7e15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:32.739407 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b43cbee-0a4d-4cd6-857b-82f6904b7e15 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:32.739733 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b43cbee-0a4d-4cd6-857b-82f6904b7e15 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:32.740011 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b43cbee-0a4d-4cd6-857b-82f6904b7e15 None admin] PATH_INFO: `/v3/projects/7d6be68cf8b046d5b03fba3c5a367c11` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:32.741167 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b43cbee-0a4d-4cd6-857b-82f6904b7e15 None admin] RBAC: Authorizing `identity:get_project(project_id=7d6be68cf8b046d5b03fba3c5a367c11)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:32.742607 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b43cbee-0a4d-4cd6-857b-82f6904b7e15 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:32.744605 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 86/168] 10.222.0.16 () {56 vars in 1186 bytes} [Wed Aug 24 09:07:32 2022] GET /identity/v3/projects/7d6be68cf8b046d5b03fba3c5a367c11 => generated 298 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:07:32.745451 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b43cbee-0a4d-4cd6-857b-82f6904b7e15 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:32.844086 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:32.846201 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b10f46-41f2-4879-9803-9e6950b3eec9 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:32.850920 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b43cbee-0a4d-4cd6-857b-82f6904b7e15 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:32.886763 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6006c8a-4360-476f-87fd-8fec1675a2d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:32.887403 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6006c8a-4360-476f-87fd-8fec1675a2d4 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:32.887711 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6006c8a-4360-476f-87fd-8fec1675a2d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:32.888016 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6006c8a-4360-476f-87fd-8fec1675a2d4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:32.900115 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6006c8a-4360-476f-87fd-8fec1675a2d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:07:32.906076 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-a6006c8a-4360-476f-87fd-8fec1675a2d4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 09:07:32.906076 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:07:32.906076 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:07:32.906076 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:07:32.906076 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:07:32.906076 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:07:32.906076 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:07:32.906076 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:07:32.906076 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:07:32.906076 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:07:32.906076 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:07:32.906076 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:07:32.906076 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:07:32.906076 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:07:32.906076 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:07:32.906076 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:07:32.906955 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:07:32.906955 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:07:32.906955 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:07:32.906955 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:07:32.906955 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:07:32.906955 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:07:32.906955 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:07:32.906955 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:07:32.906955 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:07:32.906955 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:07:32.906955 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:07:32.906955 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 24 09:07:32.906955 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:07:32.906955 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:07:32.906955 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:07:32.906955 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:07:32.907911 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:07:32.907911 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:07:32.907911 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:07:32.907911 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:07:32.907911 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:07:32.907911 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:07:32.907911 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:07:32.907911 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:07:32.907911 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:07:32.907911 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:07:32.907911 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 09:07:32.907911 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 24 09:07:32.907911 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 87/169] 10.222.0.16 () {56 vars in 1114 bytes} [Wed Aug 24 09:07:32 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:07:32.908564 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6006c8a-4360-476f-87fd-8fec1675a2d4 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:32.928443 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effe634f-71c1-41a9-ab7c-1bc856c11fb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:32.929077 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effe634f-71c1-41a9-ab7c-1bc856c11fb3 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:32.929359 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effe634f-71c1-41a9-ab7c-1bc856c11fb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:32.929637 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effe634f-71c1-41a9-ab7c-1bc856c11fb3 None admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:32.930315 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effe634f-71c1-41a9-ab7c-1bc856c11fb3 None admin] RBAC: Adding query filter params (name=member) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:07:32.930620 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effe634f-71c1-41a9-ab7c-1bc856c11fb3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:32.932206 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effe634f-71c1-41a9-ab7c-1bc856c11fb3 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:32.955916 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 83/170] 10.222.0.16 () {56 vars in 1101 bytes} [Wed Aug 24 09:07:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:07:32.956713 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effe634f-71c1-41a9-ab7c-1bc856c11fb3 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:33.019566 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6006c8a-4360-476f-87fd-8fec1675a2d4 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:33.057134 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fb0a20-621c-4b1f-8069-db492307828f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:33.057815 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fb0a20-621c-4b1f-8069-db492307828f None admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:33.058242 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fb0a20-621c-4b1f-8069-db492307828f None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:33.058526 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fb0a20-621c-4b1f-8069-db492307828f None admin] PATH_INFO: `/v3/projects/7d6be68cf8b046d5b03fba3c5a367c11/users/693fc8b7114049948366081c3e04537c/roles/5a55760968cc43ef86318e5bec24f4ef` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:33.060648 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fb0a20-621c-4b1f-8069-db492307828f None admin] RBAC: Authorizing `identity:create_grant(project_id=7d6be68cf8b046d5b03fba3c5a367c11, user_id=693fc8b7114049948366081c3e04537c, role_id=5a55760968cc43ef86318e5bec24f4ef)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:33.062111 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fb0a20-621c-4b1f-8069-db492307828f None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:33.073420 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:33.126711 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effe634f-71c1-41a9-ab7c-1bc856c11fb3 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:33.159859 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 88/171] 10.222.0.16 () {58 vars in 1439 bytes} [Wed Aug 24 09:07:32 2022] PUT /identity/v3/projects/7d6be68cf8b046d5b03fba3c5a367c11/users/693fc8b7114049948366081c3e04537c/roles/5a55760968cc43ef86318e5bec24f4ef => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:07:33.160707 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fb0a20-621c-4b1f-8069-db492307828f None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:33.188015 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:33.193195 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a21ca65-9d19-4ece-9d21-1a4a9244a5a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:33.193845 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a21ca65-9d19-4ece-9d21-1a4a9244a5a8 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:33.194131 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a21ca65-9d19-4ece-9d21-1a4a9244a5a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:33.194407 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a21ca65-9d19-4ece-9d21-1a4a9244a5a8 None admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:33.203043 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a21ca65-9d19-4ece-9d21-1a4a9244a5a8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:07:33.203043 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a21ca65-9d19-4ece-9d21-1a4a9244a5a8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:33.203043 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a21ca65-9d19-4ece-9d21-1a4a9244a5a8 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:33.218768 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 84/172] 10.222.0.16 () {56 vars in 1111 bytes} [Wed Aug 24 09:07:32 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:07:33.224548 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a21ca65-9d19-4ece-9d21-1a4a9244a5a8 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:33.381781 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fb0a20-621c-4b1f-8069-db492307828f None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:33.429787 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a21ca65-9d19-4ece-9d21-1a4a9244a5a8 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:33.452571 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a05131-90e1-46db-9851-80ab61f2b4a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:33.453209 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a05131-90e1-46db-9851-80ab61f2b4a1 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:33.453483 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a05131-90e1-46db-9851-80ab61f2b4a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:33.453752 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a05131-90e1-46db-9851-80ab61f2b4a1 None admin] PATH_INFO: `/v3/users/1925e7a247b64a06b5abece3c5687b12` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:33.454540 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:33.456926 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a05131-90e1-46db-9851-80ab61f2b4a1 None admin] RBAC: Authorizing `identity:get_user(user_id=1925e7a247b64a06b5abece3c5687b12)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:33.460131 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a05131-90e1-46db-9851-80ab61f2b4a1 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:33.465101 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 89/173] 10.222.0.16 () {56 vars in 1177 bytes} [Wed Aug 24 09:07:33 2022] GET /identity/v3/users/1925e7a247b64a06b5abece3c5687b12 => generated 275 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:07:33.469860 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a05131-90e1-46db-9851-80ab61f2b4a1 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:33.500638 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d262f68-a8e0-41d3-b3e6-c2473ded9a27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:33.501334 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d262f68-a8e0-41d3-b3e6-c2473ded9a27 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:33.501609 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d262f68-a8e0-41d3-b3e6-c2473ded9a27 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:33.501875 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d262f68-a8e0-41d3-b3e6-c2473ded9a27 None admin] PATH_INFO: `/v3/users/1925e7a247b64a06b5abece3c5687b12` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:33.505210 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d262f68-a8e0-41d3-b3e6-c2473ded9a27 None admin] RBAC: Authorizing `identity:get_user(user_id=1925e7a247b64a06b5abece3c5687b12)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:33.509110 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d262f68-a8e0-41d3-b3e6-c2473ded9a27 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:33.515140 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 85/174] 10.222.0.16 () {56 vars in 1177 bytes} [Wed Aug 24 09:07:33 2022] GET /identity/v3/users/1925e7a247b64a06b5abece3c5687b12 => generated 275 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:07:33.520729 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d262f68-a8e0-41d3-b3e6-c2473ded9a27 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:33.648673 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:33.713609 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a05131-90e1-46db-9851-80ab61f2b4a1 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:33.769342 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d262f68-a8e0-41d3-b3e6-c2473ded9a27 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:33.787387 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7155f339-1317-47a6-9c8b-9be47190387e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:33.788711 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7155f339-1317-47a6-9c8b-9be47190387e None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:33.789201 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7155f339-1317-47a6-9c8b-9be47190387e None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:33.789721 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7155f339-1317-47a6-9c8b-9be47190387e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:33.805080 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8451dfb-d3b0-4aae-a1a7-d8f3510e1523 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:33.805704 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8451dfb-d3b0-4aae-a1a7-d8f3510e1523 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:33.805988 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8451dfb-d3b0-4aae-a1a7-d8f3510e1523 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:33.806267 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8451dfb-d3b0-4aae-a1a7-d8f3510e1523 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:33.894722 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:33.933429 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7155f339-1317-47a6-9c8b-9be47190387e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:33.934909 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7155f339-1317-47a6-9c8b-9be47190387e None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:33.937628 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8451dfb-d3b0-4aae-a1a7-d8f3510e1523 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:33.939088 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8451dfb-d3b0-4aae-a1a7-d8f3510e1523 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:34.104382 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:34.126431 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 86/175] 10.222.0.16 () {58 vars in 1282 bytes} [Wed Aug 24 09:07:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 606 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 09:07:34.128583 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4b1661-e25e-4d26-a319-8d1115c3e4bb None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:34.128870 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4b1661-e25e-4d26-a319-8d1115c3e4bb None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:34.129145 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4b1661-e25e-4d26-a319-8d1115c3e4bb None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:34.131467 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 87/176] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:07:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:07:34.137268 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4b1661-e25e-4d26-a319-8d1115c3e4bb None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:34.154167 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 90/177] 10.222.0.16 () {58 vars in 1282 bytes} [Wed Aug 24 09:07:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 684 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 09:07:34.155270 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a345bbd8-0b8c-40a5-b55b-dcaf1a6c4675 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:34.155665 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a345bbd8-0b8c-40a5-b55b-dcaf1a6c4675 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:34.155859 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a345bbd8-0b8c-40a5-b55b-dcaf1a6c4675 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:34.192319 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-a345bbd8-0b8c-40a5-b55b-dcaf1a6c4675 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:07:34.193553 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-a345bbd8-0b8c-40a5-b55b-dcaf1a6c4675 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:07:34.308633 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:34.330613 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 91/178] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:07:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:07:34.331242 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a345bbd8-0b8c-40a5-b55b-dcaf1a6c4675 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:34.356248 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4b1661-e25e-4d26-a319-8d1115c3e4bb None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:34.430441 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3597251a-4112-4b1e-9de5-4b27f39dc33a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:34.431080 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3597251a-4112-4b1e-9de5-4b27f39dc33a None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:34.431461 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3597251a-4112-4b1e-9de5-4b27f39dc33a None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:34.431763 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3597251a-4112-4b1e-9de5-4b27f39dc33a None admin] PATH_INFO: `/v3/projects/7d6be68cf8b046d5b03fba3c5a367c11` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:34.436665 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3597251a-4112-4b1e-9de5-4b27f39dc33a None admin] RBAC: Authorizing `identity:get_project(project_id=7d6be68cf8b046d5b03fba3c5a367c11)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:34.438226 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3597251a-4112-4b1e-9de5-4b27f39dc33a None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:34.440317 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 88/179] 10.222.0.16 () {56 vars in 1186 bytes} [Wed Aug 24 09:07:34 2022] GET /identity/v3/projects/7d6be68cf8b046d5b03fba3c5a367c11 => generated 298 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:07:34.446031 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3597251a-4112-4b1e-9de5-4b27f39dc33a None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:34.474152 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a345bbd8-0b8c-40a5-b55b-dcaf1a6c4675 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:34.490701 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:34.543810 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3998e8dc-e39f-493a-825b-ce3e2484e805 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:34.544449 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3998e8dc-e39f-493a-825b-ce3e2484e805 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:34.544727 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3998e8dc-e39f-493a-825b-ce3e2484e805 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:34.545534 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3998e8dc-e39f-493a-825b-ce3e2484e805 None admin] PATH_INFO: `/v3/projects/d6829a2bfe5a433dbc0b7b3dc0283fdf` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:34.546713 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3998e8dc-e39f-493a-825b-ce3e2484e805 None admin] RBAC: Authorizing `identity:get_project(project_id=d6829a2bfe5a433dbc0b7b3dc0283fdf)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:34.548182 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3998e8dc-e39f-493a-825b-ce3e2484e805 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:34.556718 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 92/180] 10.222.0.16 () {56 vars in 1186 bytes} [Wed Aug 24 09:07:34 2022] GET /identity/v3/projects/d6829a2bfe5a433dbc0b7b3dc0283fdf => generated 312 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:07:34.558505 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3998e8dc-e39f-493a-825b-ce3e2484e805 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:34.660309 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3597251a-4112-4b1e-9de5-4b27f39dc33a None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:34.715465 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b03d46-7197-4516-84ee-0b0d5b015571 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:34.716472 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b03d46-7197-4516-84ee-0b0d5b015571 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:34.717489 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b03d46-7197-4516-84ee-0b0d5b015571 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:34.717959 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b03d46-7197-4516-84ee-0b0d5b015571 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:34.718885 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b03d46-7197-4516-84ee-0b0d5b015571 None admin] RBAC: Adding query filter params (role.id=ed6466d17fb7403c8491bb7f1ef95fc1, scope.project.id=7d6be68cf8b046d5b03fba3c5a367c11, user.id=1925e7a247b64a06b5abece3c5687b12) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:07:34.719412 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b03d46-7197-4516-84ee-0b0d5b015571 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:34.721181 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b03d46-7197-4516-84ee-0b0d5b015571 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:34.735627 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 89/181] 10.222.0.16 () {56 vars in 1374 bytes} [Wed Aug 24 09:07:34 2022] GET /identity/v3/role_assignments?user.id=1925e7a247b64a06b5abece3c5687b12&scope.project.id=7d6be68cf8b046d5b03fba3c5a367c11&role.id=ed6466d17fb7403c8491bb7f1ef95fc1 => generated 608 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:07:34.772013 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3998e8dc-e39f-493a-825b-ce3e2484e805 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:34.815170 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089fa703-46d8-4bf7-8a79-096188c08d82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:34.816146 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089fa703-46d8-4bf7-8a79-096188c08d82 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:34.816571 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089fa703-46d8-4bf7-8a79-096188c08d82 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:34.817063 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089fa703-46d8-4bf7-8a79-096188c08d82 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:34.817963 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089fa703-46d8-4bf7-8a79-096188c08d82 None admin] RBAC: Adding query filter params (role.id=136cdf78e00a426997dd4e37faff489b, scope.project.id=d6829a2bfe5a433dbc0b7b3dc0283fdf, user.id=1925e7a247b64a06b5abece3c5687b12) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:07:34.818444 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089fa703-46d8-4bf7-8a79-096188c08d82 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:34.820139 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089fa703-46d8-4bf7-8a79-096188c08d82 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:34.827864 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 93/182] 10.222.0.16 () {56 vars in 1374 bytes} [Wed Aug 24 09:07:34 2022] GET /identity/v3/role_assignments?user.id=1925e7a247b64a06b5abece3c5687b12&scope.project.id=d6829a2bfe5a433dbc0b7b3dc0283fdf&role.id=136cdf78e00a426997dd4e37faff489b => generated 608 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:07:34.861916 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b03d46-7197-4516-84ee-0b0d5b015571 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:34.876049 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:34.972827 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0f61e9-e407-44eb-b594-4da50470faf5 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:34.973811 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0f61e9-e407-44eb-b594-4da50470faf5 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:34.974283 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0f61e9-e407-44eb-b594-4da50470faf5 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:34.981248 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 94/183] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:07:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 09:07:34.984253 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023654a1-c885-4808-ab26-ad687529ae35 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:34.984533 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023654a1-c885-4808-ab26-ad687529ae35 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:34.984801 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023654a1-c885-4808-ab26-ad687529ae35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:35.017551 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-023654a1-c885-4808-ab26-ad687529ae35 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:07:35.019303 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-023654a1-c885-4808-ab26-ad687529ae35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:07:35.019905 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b03d46-7197-4516-84ee-0b0d5b015571 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:35.090366 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:35.098427 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5085266f-333a-4284-a6e5-73ef964436d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:35.101800 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5085266f-333a-4284-a6e5-73ef964436d3 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:35.101800 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5085266f-333a-4284-a6e5-73ef964436d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:35.101800 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5085266f-333a-4284-a6e5-73ef964436d3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:35.112494 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5085266f-333a-4284-a6e5-73ef964436d3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:07:35.117889 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-5085266f-333a-4284-a6e5-73ef964436d3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:07:35.117889 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:07:35.117889 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:07:35.117889 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:07:35.117889 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:07:35.117889 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:07:35.117889 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:07:35.117889 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:07:35.117889 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:07:35.117889 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:07:35.117889 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:07:35.117889 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:07:35.117889 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:07:35.117889 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:07:35.117889 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:07:35.117889 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:07:35.118767 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:07:35.118767 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:07:35.118767 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:07:35.118767 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:07:35.118767 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:07:35.118767 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:07:35.118767 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:07:35.118767 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:07:35.118767 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:07:35.118767 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:07:35.118767 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:07:35.118767 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Aug 24 09:07:35.118767 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:07:35.118767 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:07:35.118767 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:07:35.118767 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:07:35.120058 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:07:35.120058 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:07:35.120058 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:07:35.120058 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:07:35.120058 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:07:35.120058 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:07:35.120058 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:07:35.120058 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:07:35.120058 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:07:35.120058 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:07:35.120058 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:07:35.120058 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Aug 24 09:07:35.120058 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 90/184] 10.222.0.16 () {56 vars in 1096 bytes} [Wed Aug 24 09:07:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:07:35.125694 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5085266f-333a-4284-a6e5-73ef964436d3 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:35.126448 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 95/185] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:07:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:07:35.201553 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:35.233737 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5085266f-333a-4284-a6e5-73ef964436d3 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:35.270377 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da837bc-dfa4-4cab-94c7-4d2972883065 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:35.270999 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da837bc-dfa4-4cab-94c7-4d2972883065 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:35.271280 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da837bc-dfa4-4cab-94c7-4d2972883065 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:35.271553 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da837bc-dfa4-4cab-94c7-4d2972883065 None admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:35.272258 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da837bc-dfa4-4cab-94c7-4d2972883065 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:07:35.272564 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da837bc-dfa4-4cab-94c7-4d2972883065 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:35.273967 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da837bc-dfa4-4cab-94c7-4d2972883065 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:35.282989 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 91/186] 10.222.0.16 () {56 vars in 1099 bytes} [Wed Aug 24 09:07:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:07:35.288598 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023654a1-c885-4808-ab26-ad687529ae35 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:35.359473 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb43320a-d662-446c-932d-02d4ee0d1d6e None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:35.359795 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb43320a-d662-446c-932d-02d4ee0d1d6e None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:35.360074 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb43320a-d662-446c-932d-02d4ee0d1d6e None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:35.361689 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 92/187] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:07:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 09:07:35.367358 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5382440-fc7e-475b-a537-602fa9ce7d06 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:35.367664 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5382440-fc7e-475b-a537-602fa9ce7d06 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:35.367941 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5382440-fc7e-475b-a537-602fa9ce7d06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:35.388039 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-c5382440-fc7e-475b-a537-602fa9ce7d06 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:07:35.389308 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-c5382440-fc7e-475b-a537-602fa9ce7d06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:07:35.437446 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023654a1-c885-4808-ab26-ad687529ae35 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:35.474559 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 93/188] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:07:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:07:35.487545 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5382440-fc7e-475b-a537-602fa9ce7d06 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:35.497657 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4e4417-4a85-414d-bf43-9ec54350115c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:35.498888 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4e4417-4a85-414d-bf43-9ec54350115c None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:35.499178 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4e4417-4a85-414d-bf43-9ec54350115c None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:35.499451 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4e4417-4a85-414d-bf43-9ec54350115c None admin] PATH_INFO: `/v3/users/7ceed91937fe4c419f33b4854f0e8d6b` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:35.513592 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4e4417-4a85-414d-bf43-9ec54350115c None admin] RBAC: Authorizing `identity:get_user(user_id=7ceed91937fe4c419f33b4854f0e8d6b)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:35.515117 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4e4417-4a85-414d-bf43-9ec54350115c None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:35.517101 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 96/189] 10.222.0.16 () {56 vars in 1177 bytes} [Wed Aug 24 09:07:35 2022] GET /identity/v3/users/7ceed91937fe4c419f33b4854f0e8d6b => generated 283 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:07:35.523710 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4e4417-4a85-414d-bf43-9ec54350115c None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:35.557541 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:35.565248 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:35.597136 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5382440-fc7e-475b-a537-602fa9ce7d06 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:35.639405 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fea1532-1b58-4c39-afa0-c9cf662456a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:35.640443 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fea1532-1b58-4c39-afa0-c9cf662456a7 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:35.640873 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fea1532-1b58-4c39-afa0-c9cf662456a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:35.641287 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fea1532-1b58-4c39-afa0-c9cf662456a7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:35.653103 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fea1532-1b58-4c39-afa0-c9cf662456a7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:07:35.653408 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4e4417-4a85-414d-bf43-9ec54350115c None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:35.658954 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-6fea1532-1b58-4c39-afa0-c9cf662456a7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 09:07:35.658954 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:07:35.658954 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:07:35.658954 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:07:35.658954 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:07:35.658954 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:07:35.658954 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:07:35.658954 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:07:35.658954 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:07:35.658954 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:07:35.658954 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:07:35.658954 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:07:35.658954 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:07:35.658954 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:07:35.658954 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:07:35.658954 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:07:35.659875 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:07:35.659875 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:07:35.659875 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:07:35.659875 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:07:35.659875 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:07:35.659875 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:07:35.659875 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:07:35.659875 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:07:35.659875 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:07:35.659875 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:07:35.659875 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:07:35.659875 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Aug 24 09:07:35.659875 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:07:35.659875 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:07:35.659875 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:07:35.659875 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:07:35.660687 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:07:35.660687 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:07:35.660687 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:07:35.660687 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:07:35.660687 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:07:35.660687 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:07:35.660687 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:07:35.660687 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:07:35.660687 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:07:35.660687 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:07:35.660687 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 09:07:35.660687 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Aug 24 09:07:35.662895 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 94/190] 10.222.0.16 () {56 vars in 1114 bytes} [Wed Aug 24 09:07:35 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:07:35.671664 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fea1532-1b58-4c39-afa0-c9cf662456a7 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:35.699036 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5922f4d0-2e10-4669-80f2-b5d4eb3c1113 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:35.700186 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5922f4d0-2e10-4669-80f2-b5d4eb3c1113 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:35.700770 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5922f4d0-2e10-4669-80f2-b5d4eb3c1113 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:35.701233 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5922f4d0-2e10-4669-80f2-b5d4eb3c1113 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:35.775374 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5922f4d0-2e10-4669-80f2-b5d4eb3c1113 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:35.776780 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5922f4d0-2e10-4669-80f2-b5d4eb3c1113 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:35.780553 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fea1532-1b58-4c39-afa0-c9cf662456a7 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:35.804001 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:35.807207 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:35.815105 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2a5e3f-b940-4636-bda2-dbd78a43cab5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:35.815762 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2a5e3f-b940-4636-bda2-dbd78a43cab5 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:35.816043 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2a5e3f-b940-4636-bda2-dbd78a43cab5 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:35.816315 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2a5e3f-b940-4636-bda2-dbd78a43cab5 None admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:35.816977 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2a5e3f-b940-4636-bda2-dbd78a43cab5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:07:35.817279 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2a5e3f-b940-4636-bda2-dbd78a43cab5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:35.818658 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2a5e3f-b940-4636-bda2-dbd78a43cab5 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:35.826336 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 95/191] 10.222.0.16 () {56 vars in 1111 bytes} [Wed Aug 24 09:07:35 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:07:35.827480 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758620cd-94c9-4844-8309-4de95a972861 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:35.827790 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758620cd-94c9-4844-8309-4de95a972861 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:35.828072 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758620cd-94c9-4844-8309-4de95a972861 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:35.829266 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 96/192] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:07:35 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:07:35.831876 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758620cd-94c9-4844-8309-4de95a972861 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:35.879020 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 97/193] 10.222.0.16 () {58 vars in 1282 bytes} [Wed Aug 24 09:07:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 09:07:35.879898 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5922f4d0-2e10-4669-80f2-b5d4eb3c1113 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:35.931451 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758620cd-94c9-4844-8309-4de95a972861 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:35.964982 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0799b62-b3c1-4c7d-b07e-c6507a65f410 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:35.965611 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0799b62-b3c1-4c7d-b07e-c6507a65f410 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:35.965890 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0799b62-b3c1-4c7d-b07e-c6507a65f410 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:35.966163 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0799b62-b3c1-4c7d-b07e-c6507a65f410 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:35.967312 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0799b62-b3c1-4c7d-b07e-c6507a65f410 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:35.968817 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0799b62-b3c1-4c7d-b07e-c6507a65f410 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:35.970394 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 97/194] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 24 09:07:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:07:35.971500 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61db3bd-5454-4695-9136-1371b0bfe5c0 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:35.971809 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61db3bd-5454-4695-9136-1371b0bfe5c0 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:35.972083 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61db3bd-5454-4695-9136-1371b0bfe5c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:35.983550 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5922f4d0-2e10-4669-80f2-b5d4eb3c1113 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:35.989754 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-c61db3bd-5454-4695-9136-1371b0bfe5c0 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:07:35.990947 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-c61db3bd-5454-4695-9136-1371b0bfe5c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:07:36.012045 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:36.016848 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3c644d-3b75-4cf2-b4d0-af166760e690 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:36.017484 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3c644d-3b75-4cf2-b4d0-af166760e690 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:36.017813 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3c644d-3b75-4cf2-b4d0-af166760e690 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:36.018118 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3c644d-3b75-4cf2-b4d0-af166760e690 None admin] PATH_INFO: `/v3/users/7ceed91937fe4c419f33b4854f0e8d6b` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:36.024956 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3c644d-3b75-4cf2-b4d0-af166760e690 None admin] RBAC: Authorizing `identity:get_user(user_id=7ceed91937fe4c419f33b4854f0e8d6b)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:36.024956 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3c644d-3b75-4cf2-b4d0-af166760e690 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:36.024956 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 98/195] 10.222.0.16 () {56 vars in 1177 bytes} [Wed Aug 24 09:07:35 2022] GET /identity/v3/users/7ceed91937fe4c419f33b4854f0e8d6b => generated 283 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:07:36.024956 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3c644d-3b75-4cf2-b4d0-af166760e690 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:36.042131 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:36.062656 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 98/196] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:07:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:07:36.063357 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61db3bd-5454-4695-9136-1371b0bfe5c0 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:36.131792 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3c644d-3b75-4cf2-b4d0-af166760e690 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:36.170358 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac3f2ae-7828-432e-b080-45dd1ebe42d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:36.171280 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac3f2ae-7828-432e-b080-45dd1ebe42d6 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:36.171454 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac3f2ae-7828-432e-b080-45dd1ebe42d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:36.171865 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac3f2ae-7828-432e-b080-45dd1ebe42d6 None admin] PATH_INFO: `/v3/projects/99f7f068dc714797ac5d88928535f7a2` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:36.172050 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61db3bd-5454-4695-9136-1371b0bfe5c0 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:36.172906 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac3f2ae-7828-432e-b080-45dd1ebe42d6 None admin] RBAC: Authorizing `identity:get_project(project_id=99f7f068dc714797ac5d88928535f7a2)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:36.174620 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac3f2ae-7828-432e-b080-45dd1ebe42d6 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:36.176453 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 99/197] 10.222.0.16 () {56 vars in 1186 bytes} [Wed Aug 24 09:07:36 2022] GET /identity/v3/projects/99f7f068dc714797ac5d88928535f7a2 => generated 302 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:07:36.177129 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac3f2ae-7828-432e-b080-45dd1ebe42d6 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:36.215226 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bea3fe2-76d7-489e-a7a1-a07e0e0a3b6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:36.216171 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bea3fe2-76d7-489e-a7a1-a07e0e0a3b6e None admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:36.216653 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bea3fe2-76d7-489e-a7a1-a07e0e0a3b6e None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:36.217143 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bea3fe2-76d7-489e-a7a1-a07e0e0a3b6e None admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:36.218378 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bea3fe2-76d7-489e-a7a1-a07e0e0a3b6e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:36.220418 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bea3fe2-76d7-489e-a7a1-a07e0e0a3b6e None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:36.244784 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:36.276917 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac3f2ae-7828-432e-b080-45dd1ebe42d6 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:36.310805 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5c732c-a39a-45d7-84e6-846380a258ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:36.311416 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5c732c-a39a-45d7-84e6-846380a258ef None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:36.311728 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5c732c-a39a-45d7-84e6-846380a258ef None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:36.311998 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5c732c-a39a-45d7-84e6-846380a258ef None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:36.377044 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5c732c-a39a-45d7-84e6-846380a258ef None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:36.378502 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5c732c-a39a-45d7-84e6-846380a258ef None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:36.472534 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 100/198] 10.222.0.16 () {58 vars in 1282 bytes} [Wed Aug 24 09:07:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 09:07:36.476402 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5c732c-a39a-45d7-84e6-846380a258ef None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:36.518744 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:36.562927 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 99/199] 10.222.0.16 () {60 vars in 1132 bytes} [Wed Aug 24 09:07:36 2022] POST /identity/v3/users => generated 297 bytes in 500 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:07:36.564073 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bea3fe2-76d7-489e-a7a1-a07e0e0a3b6e None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:36.578085 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5c732c-a39a-45d7-84e6-846380a258ef None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:36.617771 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:36.621934 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70715916-25c4-459c-b5c7-53bbafca6eaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:36.622568 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70715916-25c4-459c-b5c7-53bbafca6eaf None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:36.622844 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70715916-25c4-459c-b5c7-53bbafca6eaf None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:36.623141 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70715916-25c4-459c-b5c7-53bbafca6eaf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:36.623957 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70715916-25c4-459c-b5c7-53bbafca6eaf None admin] RBAC: Adding query filter params (role.id=15f634d4a2d247ddab90c458e5e1c141, scope.project.id=99f7f068dc714797ac5d88928535f7a2, user.id=7ceed91937fe4c419f33b4854f0e8d6b) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:07:36.624266 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70715916-25c4-459c-b5c7-53bbafca6eaf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:36.625805 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70715916-25c4-459c-b5c7-53bbafca6eaf None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:36.643779 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 101/200] 10.222.0.16 () {56 vars in 1374 bytes} [Wed Aug 24 09:07:36 2022] GET /identity/v3/role_assignments?user.id=7ceed91937fe4c419f33b4854f0e8d6b&scope.project.id=99f7f068dc714797ac5d88928535f7a2&role.id=15f634d4a2d247ddab90c458e5e1c141 => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:07:36.643779 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70715916-25c4-459c-b5c7-53bbafca6eaf None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:36.687203 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:36.705210 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bea3fe2-76d7-489e-a7a1-a07e0e0a3b6e None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:36.764358 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3312a2-9919-4142-bea8-990c7d311184 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:36.765159 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3312a2-9919-4142-bea8-990c7d311184 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:36.765444 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3312a2-9919-4142-bea8-990c7d311184 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:36.765720 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3312a2-9919-4142-bea8-990c7d311184 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:36.781861 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3312a2-9919-4142-bea8-990c7d311184 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:07:36.785140 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70715916-25c4-459c-b5c7-53bbafca6eaf None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:36.788380 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-cc3312a2-9919-4142-bea8-990c7d311184 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 09:07:36.788380 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:07:36.788380 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:07:36.788380 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:07:36.788380 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:07:36.788380 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:07:36.788380 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:07:36.788380 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:07:36.788380 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:07:36.788380 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:07:36.788380 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:07:36.788380 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:07:36.788380 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:07:36.788380 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:07:36.788380 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:07:36.788380 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:07:36.789330 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:07:36.789330 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:07:36.789330 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:07:36.789330 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:07:36.789330 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:07:36.789330 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:07:36.789330 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:07:36.789330 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:07:36.789330 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:07:36.789330 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:07:36.789330 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:07:36.789330 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Aug 24 09:07:36.789330 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:07:36.789330 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:07:36.789330 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:07:36.789330 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:07:36.790196 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:07:36.790196 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:07:36.790196 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:07:36.790196 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:07:36.790196 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:07:36.790196 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:07:36.790196 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:07:36.790196 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:07:36.790196 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:07:36.790196 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:07:36.790196 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 09:07:36.790196 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Aug 24 09:07:36.790196 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 100/201] 10.222.0.16 () {56 vars in 1099 bytes} [Wed Aug 24 09:07:36 2022] GET /identity/v3/roles/reader => generated 84 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:07:36.796550 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3312a2-9919-4142-bea8-990c7d311184 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:36.829210 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025801ad-19d4-4e5c-8c29-1952fb218691 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:36.829210 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025801ad-19d4-4e5c-8c29-1952fb218691 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:36.829210 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025801ad-19d4-4e5c-8c29-1952fb218691 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:36.829210 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025801ad-19d4-4e5c-8c29-1952fb218691 None admin] PATH_INFO: `/v3/projects/99f7f068dc714797ac5d88928535f7a2` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:36.832356 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025801ad-19d4-4e5c-8c29-1952fb218691 None admin] RBAC: Authorizing `identity:get_project(project_id=99f7f068dc714797ac5d88928535f7a2)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:36.832356 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025801ad-19d4-4e5c-8c29-1952fb218691 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:36.835699 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 102/202] 10.222.0.16 () {56 vars in 1186 bytes} [Wed Aug 24 09:07:36 2022] GET /identity/v3/projects/99f7f068dc714797ac5d88928535f7a2 => generated 302 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:07:36.839656 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025801ad-19d4-4e5c-8c29-1952fb218691 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:36.923013 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:36.927805 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3312a2-9919-4142-bea8-990c7d311184 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:36.974764 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:36.974956 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04ffbd6-eadf-4b65-8e41-e2c1b415aaf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:36.975387 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04ffbd6-eadf-4b65-8e41-e2c1b415aaf8 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:36.975710 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04ffbd6-eadf-4b65-8e41-e2c1b415aaf8 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:36.975991 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04ffbd6-eadf-4b65-8e41-e2c1b415aaf8 None admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:36.977318 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04ffbd6-eadf-4b65-8e41-e2c1b415aaf8 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:07:36.977631 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04ffbd6-eadf-4b65-8e41-e2c1b415aaf8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:36.978423 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025801ad-19d4-4e5c-8c29-1952fb218691 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:36.979108 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04ffbd6-eadf-4b65-8e41-e2c1b415aaf8 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:36.990441 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 101/203] 10.222.0.16 () {56 vars in 1101 bytes} [Wed Aug 24 09:07:36 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:07:36.995556 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04ffbd6-eadf-4b65-8e41-e2c1b415aaf8 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:37.031572 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ecc31b-807b-43c8-9043-8d8c31245972 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:37.032651 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ecc31b-807b-43c8-9043-8d8c31245972 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:37.032934 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ecc31b-807b-43c8-9043-8d8c31245972 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:37.033488 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ecc31b-807b-43c8-9043-8d8c31245972 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:37.034231 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ecc31b-807b-43c8-9043-8d8c31245972 None admin] RBAC: Adding query filter params (role.id=ed6466d17fb7403c8491bb7f1ef95fc1, scope.project.id=99f7f068dc714797ac5d88928535f7a2, user.id=7ceed91937fe4c419f33b4854f0e8d6b) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:07:37.034797 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ecc31b-807b-43c8-9043-8d8c31245972 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:37.037085 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ecc31b-807b-43c8-9043-8d8c31245972 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:37.044570 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 103/204] 10.222.0.16 () {56 vars in 1374 bytes} [Wed Aug 24 09:07:36 2022] GET /identity/v3/role_assignments?user.id=7ceed91937fe4c419f33b4854f0e8d6b&scope.project.id=99f7f068dc714797ac5d88928535f7a2&role.id=ed6466d17fb7403c8491bb7f1ef95fc1 => generated 259 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:07:37.122646 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04ffbd6-eadf-4b65-8e41-e2c1b415aaf8 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:37.162617 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15fd5a3-5644-4c46-87e8-e03284076fd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:37.163286 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15fd5a3-5644-4c46-87e8-e03284076fd8 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:37.163572 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15fd5a3-5644-4c46-87e8-e03284076fd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:37.163881 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15fd5a3-5644-4c46-87e8-e03284076fd8 None admin] PATH_INFO: `/v3/users/693fc8b7114049948366081c3e04537c` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:37.165099 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15fd5a3-5644-4c46-87e8-e03284076fd8 None admin] RBAC: Authorizing `identity:get_user(user_id=693fc8b7114049948366081c3e04537c)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:37.166663 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15fd5a3-5644-4c46-87e8-e03284076fd8 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:37.168547 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 102/205] 10.222.0.16 () {56 vars in 1177 bytes} [Wed Aug 24 09:07:36 2022] GET /identity/v3/users/693fc8b7114049948366081c3e04537c => generated 289 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:07:37.175060 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ecc31b-807b-43c8-9043-8d8c31245972 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:37.286616 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ecc31b-807b-43c8-9043-8d8c31245972 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:37.324431 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1da584-3462-4f3d-8716-52567a873f96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:37.325050 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1da584-3462-4f3d-8716-52567a873f96 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:37.325338 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1da584-3462-4f3d-8716-52567a873f96 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:37.325612 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1da584-3462-4f3d-8716-52567a873f96 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:37.359967 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:37.399033 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1da584-3462-4f3d-8716-52567a873f96 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:37.400466 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1da584-3462-4f3d-8716-52567a873f96 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:37.510975 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 104/206] 10.222.0.16 () {58 vars in 1282 bytes} [Wed Aug 24 09:07:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 09:07:37.516740 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15fd5a3-5644-4c46-87e8-e03284076fd8 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:37.562180 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:37.652316 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15fd5a3-5644-4c46-87e8-e03284076fd8 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:37.688695 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c065bacc-1292-46b0-8590-b88bf2b9aec5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:37.689341 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c065bacc-1292-46b0-8590-b88bf2b9aec5 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:37.689622 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c065bacc-1292-46b0-8590-b88bf2b9aec5 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:37.689898 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c065bacc-1292-46b0-8590-b88bf2b9aec5 None admin] PATH_INFO: `/v3/projects/7d6be68cf8b046d5b03fba3c5a367c11` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:37.691041 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c065bacc-1292-46b0-8590-b88bf2b9aec5 None admin] RBAC: Authorizing `identity:get_project(project_id=7d6be68cf8b046d5b03fba3c5a367c11)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:37.692533 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c065bacc-1292-46b0-8590-b88bf2b9aec5 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:37.694524 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 103/207] 10.222.0.16 () {56 vars in 1186 bytes} [Wed Aug 24 09:07:37 2022] GET /identity/v3/projects/7d6be68cf8b046d5b03fba3c5a367c11 => generated 298 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:07:37.700018 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1da584-3462-4f3d-8716-52567a873f96 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:37.792235 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:37.826199 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1da584-3462-4f3d-8716-52567a873f96 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:37.861582 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82c5b24-8767-4b96-98ac-61689baf80c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:37.862202 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82c5b24-8767-4b96-98ac-61689baf80c3 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:37.862489 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82c5b24-8767-4b96-98ac-61689baf80c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:37.862770 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82c5b24-8767-4b96-98ac-61689baf80c3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:37.863509 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82c5b24-8767-4b96-98ac-61689baf80c3 None admin] RBAC: Adding query filter params (role.id=5a55760968cc43ef86318e5bec24f4ef, scope.project.id=7d6be68cf8b046d5b03fba3c5a367c11, user.id=693fc8b7114049948366081c3e04537c) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:07:37.863912 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82c5b24-8767-4b96-98ac-61689baf80c3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:37.866006 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82c5b24-8767-4b96-98ac-61689baf80c3 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:37.872384 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 105/208] 10.222.0.16 () {56 vars in 1374 bytes} [Wed Aug 24 09:07:37 2022] GET /identity/v3/role_assignments?user.id=693fc8b7114049948366081c3e04537c&scope.project.id=7d6be68cf8b046d5b03fba3c5a367c11&role.id=5a55760968cc43ef86318e5bec24f4ef => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:07:38.908162 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9edc1c2-6d0e-4e9f-9ff8-5de073841e90 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:38.909042 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9edc1c2-6d0e-4e9f-9ff8-5de073841e90 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:38.909479 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9edc1c2-6d0e-4e9f-9ff8-5de073841e90 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:38.911228 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 104/209] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:07:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:07:38.916804 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c4c28f-0c5a-470a-b5d8-3de25152a238 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:38.917233 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c4c28f-0c5a-470a-b5d8-3de25152a238 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:38.917425 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c4c28f-0c5a-470a-b5d8-3de25152a238 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:38.937077 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-23c4c28f-0c5a-470a-b5d8-3de25152a238 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:07:38.938601 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-23c4c28f-0c5a-470a-b5d8-3de25152a238 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:07:38.965040 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cdd76c-f2e8-44e1-9c90-c8d0ba99959c None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:38.965664 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cdd76c-f2e8-44e1-9c90-c8d0ba99959c None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:38.966169 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cdd76c-f2e8-44e1-9c90-c8d0ba99959c None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:38.967795 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 105/210] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:07:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:07:38.973032 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fc7327-6473-49f5-9ba5-37379991b0bd None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:38.973577 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fc7327-6473-49f5-9ba5-37379991b0bd None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:38.974046 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fc7327-6473-49f5-9ba5-37379991b0bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:38.992633 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-29fc7327-6473-49f5-9ba5-37379991b0bd None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:07:38.994131 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-29fc7327-6473-49f5-9ba5-37379991b0bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:07:39.016580 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 106/211] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:07:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:07:39.065974 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 106/212] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:07:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 09:07:39.117969 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3037da-a6f5-4c5e-978b-96247b6ec840 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:39.118615 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3037da-a6f5-4c5e-978b-96247b6ec840 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:39.119094 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3037da-a6f5-4c5e-978b-96247b6ec840 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:39.120684 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 107/213] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:07:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:07:39.126572 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bd1521-9289-4a55-911c-aeabacbe419a None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:39.126859 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bd1521-9289-4a55-911c-aeabacbe419a None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:39.127135 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bd1521-9289-4a55-911c-aeabacbe419a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:39.145394 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-85bd1521-9289-4a55-911c-aeabacbe419a None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:07:39.146785 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-85bd1521-9289-4a55-911c-aeabacbe419a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:07:39.167519 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:39.280647 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 107/214] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:07:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:07:39.286693 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d112560b-109d-41a1-b5e8-55375edb0d39 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:39.286984 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d112560b-109d-41a1-b5e8-55375edb0d39 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:39.287262 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d112560b-109d-41a1-b5e8-55375edb0d39 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:39.288772 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 108/215] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:07:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:07:39.299613 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af4657d-5ef7-4b7f-a136-34e2378656a7 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:39.299613 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af4657d-5ef7-4b7f-a136-34e2378656a7 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:39.299613 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af4657d-5ef7-4b7f-a136-34e2378656a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:39.316077 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d112560b-109d-41a1-b5e8-55375edb0d39 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:39.317614 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-3af4657d-5ef7-4b7f-a136-34e2378656a7 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:07:39.318950 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-3af4657d-5ef7-4b7f-a136-34e2378656a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:07:39.379089 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:39.401152 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 108/216] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:07:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:07:39.408319 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af4657d-5ef7-4b7f-a136-34e2378656a7 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:39.424425 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d112560b-109d-41a1-b5e8-55375edb0d39 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:39.463680 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d2d2ea-d065-4126-9183-3ea10bbf9fa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:39.464310 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d2d2ea-d065-4126-9183-3ea10bbf9fa7 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:39.464594 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d2d2ea-d065-4126-9183-3ea10bbf9fa7 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:39.464869 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d2d2ea-d065-4126-9183-3ea10bbf9fa7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:39.475746 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d2d2ea-d065-4126-9183-3ea10bbf9fa7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:07:39.480547 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-72d2d2ea-d065-4126-9183-3ea10bbf9fa7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 09:07:39.480547 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:07:39.480547 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:07:39.480547 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:07:39.480547 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:07:39.480547 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:07:39.480547 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:07:39.480547 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:07:39.480547 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:07:39.480547 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:07:39.480547 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:07:39.480547 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:07:39.480547 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:07:39.480547 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:07:39.480547 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:07:39.480547 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:07:39.481421 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:07:39.481421 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:07:39.481421 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:07:39.481421 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:07:39.481421 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:07:39.481421 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:07:39.481421 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:07:39.481421 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:07:39.481421 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:07:39.481421 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:07:39.481421 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:07:39.481421 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 24 09:07:39.481421 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:07:39.481421 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:07:39.481421 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:07:39.481421 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:07:39.482867 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:07:39.482867 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:07:39.482867 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:07:39.482867 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:07:39.482867 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:07:39.482867 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:07:39.482867 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:07:39.482867 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:07:39.482867 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:07:39.482867 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:07:39.482867 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 09:07:39.482867 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 24 09:07:39.482867 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 109/217] 10.222.0.16 () {56 vars in 1099 bytes} [Wed Aug 24 09:07:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 09:07:39.487309 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d2d2ea-d065-4126-9183-3ea10bbf9fa7 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:39.529402 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:39.550570 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af4657d-5ef7-4b7f-a136-34e2378656a7 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:39.586622 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29d6bd5-10ce-4c11-8e01-9d35d0f6ae9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:39.587592 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29d6bd5-10ce-4c11-8e01-9d35d0f6ae9b None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:39.587895 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29d6bd5-10ce-4c11-8e01-9d35d0f6ae9b None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:39.588174 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29d6bd5-10ce-4c11-8e01-9d35d0f6ae9b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:39.596331 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d2d2ea-d065-4126-9183-3ea10bbf9fa7 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:39.598680 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29d6bd5-10ce-4c11-8e01-9d35d0f6ae9b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:07:39.603407 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-b29d6bd5-10ce-4c11-8e01-9d35d0f6ae9b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:07:39.603407 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:07:39.603407 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:07:39.603407 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:07:39.603407 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:07:39.603407 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:07:39.603407 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:07:39.603407 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:07:39.603407 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:07:39.603407 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:07:39.603407 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:07:39.603407 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:07:39.603407 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:07:39.603407 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:07:39.603407 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:07:39.603407 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:07:39.604317 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:07:39.604317 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:07:39.604317 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:07:39.604317 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:07:39.604317 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:07:39.604317 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:07:39.604317 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:07:39.604317 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:07:39.604317 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:07:39.604317 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:07:39.604317 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:07:39.604317 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Aug 24 09:07:39.604317 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:07:39.604317 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:07:39.604317 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:07:39.604317 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:07:39.605171 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:07:39.605171 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:07:39.605171 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:07:39.605171 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:07:39.605171 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:07:39.605171 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:07:39.605171 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:07:39.605171 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:07:39.605171 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:07:39.605171 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:07:39.605171 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:07:39.605171 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Aug 24 09:07:39.605171 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 109/218] 10.222.0.16 () {56 vars in 1096 bytes} [Wed Aug 24 09:07:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:07:39.605817 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29d6bd5-10ce-4c11-8e01-9d35d0f6ae9b None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:39.619973 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:39.643404 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9bf5db-6f0b-4071-a3c0-0acbe4f33b04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:39.644305 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9bf5db-6f0b-4071-a3c0-0acbe4f33b04 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:39.644812 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9bf5db-6f0b-4071-a3c0-0acbe4f33b04 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:39.645305 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9bf5db-6f0b-4071-a3c0-0acbe4f33b04 None admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:39.646137 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9bf5db-6f0b-4071-a3c0-0acbe4f33b04 None admin] RBAC: Adding query filter params (name=member) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:07:39.646637 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9bf5db-6f0b-4071-a3c0-0acbe4f33b04 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:39.648878 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9bf5db-6f0b-4071-a3c0-0acbe4f33b04 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:39.658695 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 110/219] 10.222.0.16 () {56 vars in 1101 bytes} [Wed Aug 24 09:07:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:07:39.660159 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9bf5db-6f0b-4071-a3c0-0acbe4f33b04 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:39.712626 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29d6bd5-10ce-4c11-8e01-9d35d0f6ae9b None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:39.745740 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf7e259-3570-45a7-a48d-f57f2330ed6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:39.746360 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf7e259-3570-45a7-a48d-f57f2330ed6a None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:39.746634 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf7e259-3570-45a7-a48d-f57f2330ed6a None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:39.746900 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf7e259-3570-45a7-a48d-f57f2330ed6a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:39.748571 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf7e259-3570-45a7-a48d-f57f2330ed6a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:39.750004 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf7e259-3570-45a7-a48d-f57f2330ed6a None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:39.751539 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 110/220] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 24 09:07:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:07:39.752315 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf7e259-3570-45a7-a48d-f57f2330ed6a None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:39.764436 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9bf5db-6f0b-4071-a3c0-0acbe4f33b04 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:39.766836 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:39.798111 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479adafc-5840-42e8-ace1-07c4c3b42092 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:39.798715 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479adafc-5840-42e8-ace1-07c4c3b42092 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:39.799001 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479adafc-5840-42e8-ace1-07c4c3b42092 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:39.799277 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479adafc-5840-42e8-ace1-07c4c3b42092 None admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:39.799974 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479adafc-5840-42e8-ace1-07c4c3b42092 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:07:39.800277 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479adafc-5840-42e8-ace1-07c4c3b42092 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:39.801665 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479adafc-5840-42e8-ace1-07c4c3b42092 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:39.809398 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 111/221] 10.222.0.16 () {56 vars in 1099 bytes} [Wed Aug 24 09:07:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:07:39.810763 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479adafc-5840-42e8-ace1-07c4c3b42092 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:39.851718 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf7e259-3570-45a7-a48d-f57f2330ed6a None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:39.855252 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:39.885795 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37138c5a-40d4-42b8-960c-cda12a4dd10a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:39.886418 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37138c5a-40d4-42b8-960c-cda12a4dd10a None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:39.886693 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37138c5a-40d4-42b8-960c-cda12a4dd10a None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:39.886960 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37138c5a-40d4-42b8-960c-cda12a4dd10a None admin] PATH_INFO: `/v3/users/04d67adcc15b4c1496b767b46aaf9779` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:39.901666 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37138c5a-40d4-42b8-960c-cda12a4dd10a None admin] RBAC: Authorizing `identity:get_user(user_id=04d67adcc15b4c1496b767b46aaf9779)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:39.903131 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37138c5a-40d4-42b8-960c-cda12a4dd10a None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:39.904813 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 111/222] 10.222.0.16 () {56 vars in 1177 bytes} [Wed Aug 24 09:07:39 2022] GET /identity/v3/users/04d67adcc15b4c1496b767b46aaf9779 => generated 297 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:07:39.905565 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37138c5a-40d4-42b8-960c-cda12a4dd10a None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:39.910642 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479adafc-5840-42e8-ace1-07c4c3b42092 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:39.944067 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7527651-08b7-4c79-a04b-1c83a6edf544 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:39.944696 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7527651-08b7-4c79-a04b-1c83a6edf544 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:39.944962 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7527651-08b7-4c79-a04b-1c83a6edf544 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:39.945231 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7527651-08b7-4c79-a04b-1c83a6edf544 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:39.955682 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7527651-08b7-4c79-a04b-1c83a6edf544 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:07:39.960811 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-a7527651-08b7-4c79-a04b-1c83a6edf544 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 09:07:39.960811 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:07:39.960811 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:07:39.960811 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:07:39.960811 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:07:39.960811 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:07:39.960811 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:07:39.960811 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:07:39.960811 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:07:39.960811 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:07:39.960811 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:07:39.960811 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:07:39.960811 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:07:39.960811 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:07:39.960811 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:07:39.960811 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:07:39.961707 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:07:39.961707 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:07:39.961707 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:07:39.961707 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:07:39.961707 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:07:39.961707 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:07:39.961707 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:07:39.961707 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:07:39.961707 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:07:39.961707 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:07:39.961707 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:07:39.961707 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 24 09:07:39.961707 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:07:39.961707 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:07:39.961707 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:07:39.961707 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:07:39.962749 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:07:39.962749 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:07:39.962749 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:07:39.962749 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:07:39.962749 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:07:39.962749 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:07:39.962749 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:07:39.962749 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:07:39.962749 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:07:39.962749 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:07:39.962749 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 09:07:39.962749 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 24 09:07:39.962749 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 112/223] 10.222.0.16 () {56 vars in 1114 bytes} [Wed Aug 24 09:07:39 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:07:39.962749 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7527651-08b7-4c79-a04b-1c83a6edf544 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:39.977924 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:40.008270 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37138c5a-40d4-42b8-960c-cda12a4dd10a None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:40.041342 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3288eaa7-524b-404e-9a00-5da9f791afed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:40.041949 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3288eaa7-524b-404e-9a00-5da9f791afed None admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:40.042227 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3288eaa7-524b-404e-9a00-5da9f791afed None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:40.042502 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3288eaa7-524b-404e-9a00-5da9f791afed None admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:40.043143 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3288eaa7-524b-404e-9a00-5da9f791afed None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:40.045151 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3288eaa7-524b-404e-9a00-5da9f791afed None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:40.059419 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7527651-08b7-4c79-a04b-1c83a6edf544 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:40.060961 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:40.092933 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41607f79-ca47-43ba-a725-109dcdc6e4a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:40.093604 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41607f79-ca47-43ba-a725-109dcdc6e4a3 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:40.093884 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41607f79-ca47-43ba-a725-109dcdc6e4a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:40.094160 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41607f79-ca47-43ba-a725-109dcdc6e4a3 None admin] PATH_INFO: `/v3/users/7ceed91937fe4c419f33b4854f0e8d6b` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:40.095294 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41607f79-ca47-43ba-a725-109dcdc6e4a3 None admin] RBAC: Authorizing `identity:get_user(user_id=7ceed91937fe4c419f33b4854f0e8d6b)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:40.096785 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41607f79-ca47-43ba-a725-109dcdc6e4a3 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:40.098351 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 113/224] 10.222.0.16 () {56 vars in 1177 bytes} [Wed Aug 24 09:07:39 2022] GET /identity/v3/users/7ceed91937fe4c419f33b4854f0e8d6b => generated 283 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:07:40.099115 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41607f79-ca47-43ba-a725-109dcdc6e4a3 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:40.122882 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 112/225] 10.222.0.16 () {60 vars in 1132 bytes} [Wed Aug 24 09:07:39 2022] POST /identity/v3/users => generated 297 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:07:40.123776 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3288eaa7-524b-404e-9a00-5da9f791afed None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:40.214270 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41607f79-ca47-43ba-a725-109dcdc6e4a3 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:40.252847 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fded8eb8-9a0c-4525-a5d2-b830fa22e6d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:40.253469 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fded8eb8-9a0c-4525-a5d2-b830fa22e6d1 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:40.253745 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fded8eb8-9a0c-4525-a5d2-b830fa22e6d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:40.254016 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fded8eb8-9a0c-4525-a5d2-b830fa22e6d1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:40.257159 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3288eaa7-524b-404e-9a00-5da9f791afed None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:40.290075 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:40.301029 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88ea677-3c9a-485e-b037-291811cbb340 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:40.301656 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88ea677-3c9a-485e-b037-291811cbb340 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:40.301950 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88ea677-3c9a-485e-b037-291811cbb340 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:40.310231 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88ea677-3c9a-485e-b037-291811cbb340 None admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:40.310868 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:40.310946 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88ea677-3c9a-485e-b037-291811cbb340 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:07:40.310946 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88ea677-3c9a-485e-b037-291811cbb340 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:40.310946 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88ea677-3c9a-485e-b037-291811cbb340 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:40.315414 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 113/226] 10.222.0.16 () {56 vars in 1111 bytes} [Wed Aug 24 09:07:40 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:07:40.316281 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88ea677-3c9a-485e-b037-291811cbb340 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:40.338765 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fded8eb8-9a0c-4525-a5d2-b830fa22e6d1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:40.340999 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fded8eb8-9a0c-4525-a5d2-b830fa22e6d1 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:40.451629 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88ea677-3c9a-485e-b037-291811cbb340 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:40.471276 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 114/227] 10.222.0.16 () {58 vars in 1282 bytes} [Wed Aug 24 09:07:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 09:07:40.475691 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fded8eb8-9a0c-4525-a5d2-b830fa22e6d1 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:40.498825 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5daf05-ae16-4715-9f53-2945322bc847 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:40.499849 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5daf05-ae16-4715-9f53-2945322bc847 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:40.500128 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5daf05-ae16-4715-9f53-2945322bc847 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:40.500430 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5daf05-ae16-4715-9f53-2945322bc847 None admin] PATH_INFO: `/v3/projects/99f7f068dc714797ac5d88928535f7a2` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:40.502074 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5daf05-ae16-4715-9f53-2945322bc847 None admin] RBAC: Authorizing `identity:get_project(project_id=99f7f068dc714797ac5d88928535f7a2)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:40.504418 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5daf05-ae16-4715-9f53-2945322bc847 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:40.506483 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 114/228] 10.222.0.16 () {56 vars in 1186 bytes} [Wed Aug 24 09:07:40 2022] GET /identity/v3/projects/99f7f068dc714797ac5d88928535f7a2 => generated 302 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:07:40.507431 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5daf05-ae16-4715-9f53-2945322bc847 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:40.565527 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:40.595989 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:40.605793 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fded8eb8-9a0c-4525-a5d2-b830fa22e6d1 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:40.619659 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5daf05-ae16-4715-9f53-2945322bc847 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:40.653502 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c014e9c-f7c4-4eec-bfe5-3bfeca8cb72b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:40.654181 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c014e9c-f7c4-4eec-bfe5-3bfeca8cb72b None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:40.654669 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c014e9c-f7c4-4eec-bfe5-3bfeca8cb72b None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:40.654952 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c014e9c-f7c4-4eec-bfe5-3bfeca8cb72b None admin] PATH_INFO: `/v3/users/7ceed91937fe4c419f33b4854f0e8d6b` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:40.657009 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c014e9c-f7c4-4eec-bfe5-3bfeca8cb72b None admin] RBAC: Authorizing `identity:get_user(user_id=7ceed91937fe4c419f33b4854f0e8d6b)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:40.658471 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c014e9c-f7c4-4eec-bfe5-3bfeca8cb72b None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:40.660113 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 115/229] 10.222.0.16 () {56 vars in 1177 bytes} [Wed Aug 24 09:07:40 2022] GET /identity/v3/users/7ceed91937fe4c419f33b4854f0e8d6b => generated 283 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:07:40.660923 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c014e9c-f7c4-4eec-bfe5-3bfeca8cb72b None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:40.663081 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7d06ae-a68d-4231-8e2f-82da7a29cea3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:40.664182 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7d06ae-a68d-4231-8e2f-82da7a29cea3 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:40.665555 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7d06ae-a68d-4231-8e2f-82da7a29cea3 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:40.666051 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7d06ae-a68d-4231-8e2f-82da7a29cea3 None admin] PATH_INFO: `/v3/projects/99f7f068dc714797ac5d88928535f7a2` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:40.667433 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7d06ae-a68d-4231-8e2f-82da7a29cea3 None admin] RBAC: Authorizing `identity:get_project(project_id=99f7f068dc714797ac5d88928535f7a2)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:40.669201 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7d06ae-a68d-4231-8e2f-82da7a29cea3 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:40.672722 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 115/230] 10.222.0.16 () {56 vars in 1186 bytes} [Wed Aug 24 09:07:40 2022] GET /identity/v3/projects/99f7f068dc714797ac5d88928535f7a2 => generated 302 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:07:40.673851 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7d06ae-a68d-4231-8e2f-82da7a29cea3 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:40.773640 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c014e9c-f7c4-4eec-bfe5-3bfeca8cb72b None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:40.801012 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7d06ae-a68d-4231-8e2f-82da7a29cea3 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:40.820936 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0960d2-b34c-4dd4-8df2-9911f3c496ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:40.821961 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0960d2-b34c-4dd4-8df2-9911f3c496ad None admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:40.822441 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0960d2-b34c-4dd4-8df2-9911f3c496ad None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:40.822923 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0960d2-b34c-4dd4-8df2-9911f3c496ad None admin] PATH_INFO: `/v3/projects/99f7f068dc714797ac5d88928535f7a2/users/7ceed91937fe4c419f33b4854f0e8d6b/roles/15f634d4a2d247ddab90c458e5e1c141` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:40.826685 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0960d2-b34c-4dd4-8df2-9911f3c496ad None admin] RBAC: Authorizing `identity:create_grant(project_id=99f7f068dc714797ac5d88928535f7a2, user_id=7ceed91937fe4c419f33b4854f0e8d6b, role_id=15f634d4a2d247ddab90c458e5e1c141)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:40.828203 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0960d2-b34c-4dd4-8df2-9911f3c496ad None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:40.832455 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:40.841149 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec1a279-a2b9-4c12-9b7d-b6eb0d88236e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:40.842395 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec1a279-a2b9-4c12-9b7d-b6eb0d88236e None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:40.842737 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec1a279-a2b9-4c12-9b7d-b6eb0d88236e None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:40.843074 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec1a279-a2b9-4c12-9b7d-b6eb0d88236e None admin] PATH_INFO: `/v3/projects/99f7f068dc714797ac5d88928535f7a2` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:40.844381 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec1a279-a2b9-4c12-9b7d-b6eb0d88236e None admin] RBAC: Authorizing `identity:get_project(project_id=99f7f068dc714797ac5d88928535f7a2)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:40.845878 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec1a279-a2b9-4c12-9b7d-b6eb0d88236e None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:40.847494 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 116/231] 10.222.0.16 () {56 vars in 1186 bytes} [Wed Aug 24 09:07:40 2022] GET /identity/v3/projects/99f7f068dc714797ac5d88928535f7a2 => generated 302 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:07:40.848455 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec1a279-a2b9-4c12-9b7d-b6eb0d88236e None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:40.913039 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:40.921218 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 116/232] 10.222.0.16 () {58 vars in 1439 bytes} [Wed Aug 24 09:07:40 2022] PUT /identity/v3/projects/99f7f068dc714797ac5d88928535f7a2/users/7ceed91937fe4c419f33b4854f0e8d6b/roles/15f634d4a2d247ddab90c458e5e1c141 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:07:40.921750 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0960d2-b34c-4dd4-8df2-9911f3c496ad None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:40.963698 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec1a279-a2b9-4c12-9b7d-b6eb0d88236e None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:41.021756 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867e4d45-7ae7-470a-afd4-2eb0b7d7f1aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:41.022429 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867e4d45-7ae7-470a-afd4-2eb0b7d7f1aa None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:41.022660 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867e4d45-7ae7-470a-afd4-2eb0b7d7f1aa None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:41.022906 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867e4d45-7ae7-470a-afd4-2eb0b7d7f1aa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:41.023732 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867e4d45-7ae7-470a-afd4-2eb0b7d7f1aa None admin] RBAC: Adding query filter params (role.id=136cdf78e00a426997dd4e37faff489b, scope.project.id=99f7f068dc714797ac5d88928535f7a2, user.id=04d67adcc15b4c1496b767b46aaf9779) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:07:41.023987 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867e4d45-7ae7-470a-afd4-2eb0b7d7f1aa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:41.025440 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867e4d45-7ae7-470a-afd4-2eb0b7d7f1aa None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:41.043369 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 117/233] 10.222.0.16 () {56 vars in 1374 bytes} [Wed Aug 24 09:07:40 2022] GET /identity/v3/role_assignments?user.id=04d67adcc15b4c1496b767b46aaf9779&scope.project.id=99f7f068dc714797ac5d88928535f7a2&role.id=136cdf78e00a426997dd4e37faff489b => generated 259 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:07:41.056005 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0960d2-b34c-4dd4-8df2-9911f3c496ad None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:41.096721 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfbf8e7-a29c-4216-9d7d-85c947bf4d91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:41.097406 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfbf8e7-a29c-4216-9d7d-85c947bf4d91 None admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:41.097688 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfbf8e7-a29c-4216-9d7d-85c947bf4d91 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:41.097964 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfbf8e7-a29c-4216-9d7d-85c947bf4d91 None admin] PATH_INFO: `/v3/projects/99f7f068dc714797ac5d88928535f7a2/users/7ceed91937fe4c419f33b4854f0e8d6b/roles/ed6466d17fb7403c8491bb7f1ef95fc1` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:41.100230 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfbf8e7-a29c-4216-9d7d-85c947bf4d91 None admin] RBAC: Authorizing `identity:create_grant(project_id=99f7f068dc714797ac5d88928535f7a2, user_id=7ceed91937fe4c419f33b4854f0e8d6b, role_id=ed6466d17fb7403c8491bb7f1ef95fc1)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:41.101747 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfbf8e7-a29c-4216-9d7d-85c947bf4d91 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:41.209826 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 117/234] 10.222.0.16 () {58 vars in 1439 bytes} [Wed Aug 24 09:07:40 2022] PUT /identity/v3/projects/99f7f068dc714797ac5d88928535f7a2/users/7ceed91937fe4c419f33b4854f0e8d6b/roles/ed6466d17fb7403c8491bb7f1ef95fc1 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:07:42.375530 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ea308f-6a21-4c9a-a1c5-0cafb292cfaf None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:42.376075 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ea308f-6a21-4c9a-a1c5-0cafb292cfaf None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:42.376168 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ea308f-6a21-4c9a-a1c5-0cafb292cfaf None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:42.377692 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 118/235] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:07:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:07:42.400126 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d4e60e-8c9e-410a-8506-bc62ae1e00b3 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:42.400586 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d4e60e-8c9e-410a-8506-bc62ae1e00b3 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:42.400773 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d4e60e-8c9e-410a-8506-bc62ae1e00b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:42.426666 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-58d4e60e-8c9e-410a-8506-bc62ae1e00b3 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:07:42.427916 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-58d4e60e-8c9e-410a-8506-bc62ae1e00b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:07:42.499019 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:42.523748 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 118/236] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:07:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:07:42.629646 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81da82fb-0d50-4da8-b3ae-22ef0b101fc2 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:42.629936 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81da82fb-0d50-4da8-b3ae-22ef0b101fc2 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:42.630125 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81da82fb-0d50-4da8-b3ae-22ef0b101fc2 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:42.631677 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 119/237] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:07:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:07:42.636696 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ac22ee-554f-4b4d-8f0e-53cb9001f152 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:42.636987 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ac22ee-554f-4b4d-8f0e-53cb9001f152 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:42.637261 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ac22ee-554f-4b4d-8f0e-53cb9001f152 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:42.664614 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-40ac22ee-554f-4b4d-8f0e-53cb9001f152 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:07:42.666140 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-40ac22ee-554f-4b4d-8f0e-53cb9001f152 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:07:42.753346 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 119/238] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:07:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:07:42.928616 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81da82fb-0d50-4da8-b3ae-22ef0b101fc2 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:43.002281 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:43.050950 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346a861e-14d0-4b18-b5c2-3967abb016ce None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:43.051231 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346a861e-14d0-4b18-b5c2-3967abb016ce None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:43.051503 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346a861e-14d0-4b18-b5c2-3967abb016ce None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:43.053131 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 120/239] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:07:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:07:43.054446 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81da82fb-0d50-4da8-b3ae-22ef0b101fc2 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:43.064103 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bccdf3-b411-4c64-9623-ab0d9ebf757d None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:43.064383 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bccdf3-b411-4c64-9623-ab0d9ebf757d None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:43.064655 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bccdf3-b411-4c64-9623-ab0d9ebf757d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:43.092785 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61da0a53-ba79-41b5-bd36-ad1b2f1dcd01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:43.093762 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61da0a53-ba79-41b5-bd36-ad1b2f1dcd01 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:43.094320 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61da0a53-ba79-41b5-bd36-ad1b2f1dcd01 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:43.094795 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61da0a53-ba79-41b5-bd36-ad1b2f1dcd01 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:43.104537 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-49bccdf3-b411-4c64-9623-ab0d9ebf757d None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:07:43.105722 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-49bccdf3-b411-4c64-9623-ab0d9ebf757d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:07:43.106373 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61da0a53-ba79-41b5-bd36-ad1b2f1dcd01 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:07:43.113692 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-61da0a53-ba79-41b5-bd36-ad1b2f1dcd01 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 09:07:43.113692 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:07:43.113692 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:07:43.113692 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:07:43.113692 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:07:43.113692 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:07:43.113692 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:07:43.113692 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:07:43.113692 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:07:43.113692 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:07:43.113692 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:07:43.113692 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:07:43.113692 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:07:43.113692 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:07:43.113692 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:07:43.113692 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:07:43.114629 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:07:43.114629 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:07:43.114629 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:07:43.114629 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:07:43.114629 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:07:43.114629 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:07:43.114629 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:07:43.114629 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:07:43.114629 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:07:43.114629 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:07:43.114629 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:07:43.114629 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Aug 24 09:07:43.114629 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:07:43.114629 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:07:43.114629 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:07:43.114629 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:07:43.115466 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:07:43.115466 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:07:43.115466 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:07:43.115466 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:07:43.115466 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:07:43.115466 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:07:43.115466 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:07:43.115466 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:07:43.115466 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:07:43.115466 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:07:43.115466 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 09:07:43.115466 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Aug 24 09:07:43.118025 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 120/240] 10.222.0.16 () {56 vars in 1099 bytes} [Wed Aug 24 09:07:42 2022] GET /identity/v3/roles/reader => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:07:43.129377 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61da0a53-ba79-41b5-bd36-ad1b2f1dcd01 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:43.205361 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 121/241] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:07:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:07:43.205978 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bccdf3-b411-4c64-9623-ab0d9ebf757d None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:43.269388 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:43.327731 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bccdf3-b411-4c64-9623-ab0d9ebf757d None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:43.384182 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be20c1e-d5a8-4e1f-9b22-0619928546b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:43.384828 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be20c1e-d5a8-4e1f-9b22-0619928546b1 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:43.385099 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be20c1e-d5a8-4e1f-9b22-0619928546b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:43.385363 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be20c1e-d5a8-4e1f-9b22-0619928546b1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:43.388078 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be20c1e-d5a8-4e1f-9b22-0619928546b1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:43.390058 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be20c1e-d5a8-4e1f-9b22-0619928546b1 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:43.400772 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 122/242] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 24 09:07:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:07:43.401588 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be20c1e-d5a8-4e1f-9b22-0619928546b1 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:43.417335 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61da0a53-ba79-41b5-bd36-ad1b2f1dcd01 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:43.441870 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:43.466321 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306b92ee-b34c-4205-9f82-be53d5230118 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:43.466855 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306b92ee-b34c-4205-9f82-be53d5230118 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:43.467185 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306b92ee-b34c-4205-9f82-be53d5230118 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:43.467459 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306b92ee-b34c-4205-9f82-be53d5230118 None admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:43.468150 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306b92ee-b34c-4205-9f82-be53d5230118 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:07:43.468450 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306b92ee-b34c-4205-9f82-be53d5230118 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:43.469846 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306b92ee-b34c-4205-9f82-be53d5230118 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:43.487658 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 121/243] 10.222.0.16 () {56 vars in 1101 bytes} [Wed Aug 24 09:07:43 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 358 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:07:43.488986 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6113883-1dad-4cf1-8af4-f658538c6c52 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:43.489320 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6113883-1dad-4cf1-8af4-f658538c6c52 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:43.489600 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6113883-1dad-4cf1-8af4-f658538c6c52 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:43.490806 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 122/244] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:07:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:07:43.495867 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e171cf1-084d-4a0b-8981-bca0311f70bb None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:43.496150 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e171cf1-084d-4a0b-8981-bca0311f70bb None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:43.496431 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e171cf1-084d-4a0b-8981-bca0311f70bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:43.517011 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-1e171cf1-084d-4a0b-8981-bca0311f70bb None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:07:43.518471 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-1e171cf1-084d-4a0b-8981-bca0311f70bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:07:43.564700 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be20c1e-d5a8-4e1f-9b22-0619928546b1 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:43.582338 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:43.588959 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 123/245] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:07:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 09:07:43.589839 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e171cf1-084d-4a0b-8981-bca0311f70bb None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:43.621672 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70da2c22-b7dd-4bf9-a062-1a3be0596383 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:43.622290 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70da2c22-b7dd-4bf9-a062-1a3be0596383 None admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:43.622603 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70da2c22-b7dd-4bf9-a062-1a3be0596383 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:43.622876 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70da2c22-b7dd-4bf9-a062-1a3be0596383 None admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:43.623542 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70da2c22-b7dd-4bf9-a062-1a3be0596383 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:43.625019 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70da2c22-b7dd-4bf9-a062-1a3be0596383 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:43.709529 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e171cf1-084d-4a0b-8981-bca0311f70bb None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:43.722173 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:43.740134 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 123/246] 10.222.0.16 () {60 vars in 1132 bytes} [Wed Aug 24 09:07:43 2022] POST /identity/v3/users => generated 293 bytes in 339 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:07:43.741221 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70da2c22-b7dd-4bf9-a062-1a3be0596383 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:43.755740 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f09dbc-8122-4c1d-8572-212d44029a91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:43.756782 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f09dbc-8122-4c1d-8572-212d44029a91 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:43.757377 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f09dbc-8122-4c1d-8572-212d44029a91 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:43.757872 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f09dbc-8122-4c1d-8572-212d44029a91 None admin] PATH_INFO: `/v3/users/6a5fc2cfb6c44b4cb49a33930d70d857` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:43.782786 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f09dbc-8122-4c1d-8572-212d44029a91 None admin] RBAC: Authorizing `identity:get_user(user_id=6a5fc2cfb6c44b4cb49a33930d70d857)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:43.786135 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f09dbc-8122-4c1d-8572-212d44029a91 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:43.795984 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 124/247] 10.222.0.16 () {56 vars in 1177 bytes} [Wed Aug 24 09:07:43 2022] GET /identity/v3/users/6a5fc2cfb6c44b4cb49a33930d70d857 => generated 297 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:07:43.799641 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f09dbc-8122-4c1d-8572-212d44029a91 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:43.912761 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70da2c22-b7dd-4bf9-a062-1a3be0596383 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:43.943798 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f09dbc-8122-4c1d-8572-212d44029a91 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:43.954216 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2d418e-8023-45b4-b0ee-0a2b1e1145f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:43.955168 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2d418e-8023-45b4-b0ee-0a2b1e1145f7 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:43.955451 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2d418e-8023-45b4-b0ee-0a2b1e1145f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:43.955851 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2d418e-8023-45b4-b0ee-0a2b1e1145f7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:43.968042 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2d418e-8023-45b4-b0ee-0a2b1e1145f7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:07:43.973392 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-7a2d418e-8023-45b4-b0ee-0a2b1e1145f7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:07:43.973392 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:07:43.973392 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:07:43.973392 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:07:43.973392 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:07:43.973392 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:07:43.973392 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:07:43.973392 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:07:43.973392 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:07:43.973392 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:07:43.973392 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:07:43.973392 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:07:43.973392 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:07:43.973392 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:07:43.973392 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:07:43.973392 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:07:43.974274 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:07:43.974274 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:07:43.974274 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:07:43.974274 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:07:43.974274 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:07:43.974274 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:07:43.974274 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:07:43.974274 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:07:43.974274 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:07:43.974274 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:07:43.974274 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:07:43.974274 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 24 09:07:43.974274 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:07:43.974274 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:07:43.974274 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:07:43.974274 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:07:43.976063 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:07:43.976063 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:07:43.976063 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:07:43.976063 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:07:43.976063 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:07:43.976063 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:07:43.976063 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:07:43.976063 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:07:43.976063 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:07:43.976063 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:07:43.976063 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:07:43.976063 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 24 09:07:43.976063 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 124/248] 10.222.0.16 () {56 vars in 1096 bytes} [Wed Aug 24 09:07:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:07:43.976063 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda887cf-2451-4b76-8550-d400ec076461 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:43.976790 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda887cf-2451-4b76-8550-d400ec076461 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:43.977019 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda887cf-2451-4b76-8550-d400ec076461 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:43.978990 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 125/249] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:07:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:07:43.978990 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda887cf-2451-4b76-8550-d400ec076461 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:43.988974 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f9515a-0938-4dff-bf66-14c7e0089c9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:43.988974 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f9515a-0938-4dff-bf66-14c7e0089c9c None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:43.988974 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f9515a-0938-4dff-bf66-14c7e0089c9c None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:43.989332 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f9515a-0938-4dff-bf66-14c7e0089c9c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:44.019936 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:44.034478 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:44.074969 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f9515a-0938-4dff-bf66-14c7e0089c9c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:44.077324 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f9515a-0938-4dff-bf66-14c7e0089c9c None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:44.108611 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda887cf-2451-4b76-8550-d400ec076461 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:44.151894 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6895d994-dc01-40d7-be3b-6caa7b535bd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:44.152837 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6895d994-dc01-40d7-be3b-6caa7b535bd1 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:44.153358 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6895d994-dc01-40d7-be3b-6caa7b535bd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:44.153838 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6895d994-dc01-40d7-be3b-6caa7b535bd1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:44.165476 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6895d994-dc01-40d7-be3b-6caa7b535bd1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:07:44.171153 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-6895d994-dc01-40d7-be3b-6caa7b535bd1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 09:07:44.171153 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:07:44.171153 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:07:44.171153 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:07:44.171153 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:07:44.171153 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:07:44.171153 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:07:44.171153 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:07:44.171153 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:07:44.171153 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:07:44.171153 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:07:44.171153 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:07:44.171153 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:07:44.171153 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:07:44.171153 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:07:44.171153 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:07:44.172099 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:07:44.172099 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:07:44.172099 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:07:44.172099 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:07:44.172099 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:07:44.172099 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:07:44.172099 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:07:44.172099 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:07:44.172099 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:07:44.172099 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:07:44.172099 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:07:44.172099 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 24 09:07:44.172099 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:07:44.172099 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:07:44.172099 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:07:44.172099 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:07:44.172894 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:07:44.172894 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:07:44.172894 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:07:44.172894 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:07:44.172894 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:07:44.172894 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:07:44.172894 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:07:44.172894 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:07:44.172894 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:07:44.172894 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:07:44.172894 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 09:07:44.172894 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 24 09:07:44.175568 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 126/250] 10.222.0.16 () {56 vars in 1099 bytes} [Wed Aug 24 09:07:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:07:44.176556 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6895d994-dc01-40d7-be3b-6caa7b535bd1 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:44.198936 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 125/251] 10.222.0.16 () {58 vars in 1282 bytes} [Wed Aug 24 09:07:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 09:07:44.204369 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427b35c2-4ba5-438e-befb-221e7d359f78 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:44.205252 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427b35c2-4ba5-438e-befb-221e7d359f78 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:44.205528 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427b35c2-4ba5-438e-befb-221e7d359f78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:44.224527 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-427b35c2-4ba5-438e-befb-221e7d359f78 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:07:44.225628 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-427b35c2-4ba5-438e-befb-221e7d359f78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:07:44.299823 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:44.301875 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:44.309135 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 126/252] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:07:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:07:44.319881 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427b35c2-4ba5-438e-befb-221e7d359f78 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:44.320003 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6895d994-dc01-40d7-be3b-6caa7b535bd1 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:44.363087 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac2995d-57a5-41cd-8360-0391a82b6d9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:44.363872 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac2995d-57a5-41cd-8360-0391a82b6d9c None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:44.364208 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac2995d-57a5-41cd-8360-0391a82b6d9c None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:44.364486 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac2995d-57a5-41cd-8360-0391a82b6d9c None admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:44.365161 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac2995d-57a5-41cd-8360-0391a82b6d9c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:07:44.366655 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac2995d-57a5-41cd-8360-0391a82b6d9c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:44.368045 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac2995d-57a5-41cd-8360-0391a82b6d9c None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:44.390378 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 127/253] 10.222.0.16 () {56 vars in 1099 bytes} [Wed Aug 24 09:07:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:07:44.393983 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac2995d-57a5-41cd-8360-0391a82b6d9c None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:44.491085 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427b35c2-4ba5-438e-befb-221e7d359f78 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:44.561413 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94af213-0b86-46ee-b57d-a437681f4c4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:44.562075 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94af213-0b86-46ee-b57d-a437681f4c4d None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:44.562358 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94af213-0b86-46ee-b57d-a437681f4c4d None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:44.562630 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94af213-0b86-46ee-b57d-a437681f4c4d None admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:44.563297 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94af213-0b86-46ee-b57d-a437681f4c4d None admin] RBAC: Adding query filter params (name=member) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:07:44.564283 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94af213-0b86-46ee-b57d-a437681f4c4d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:44.567512 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94af213-0b86-46ee-b57d-a437681f4c4d None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:44.584265 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 127/254] 10.222.0.16 () {56 vars in 1101 bytes} [Wed Aug 24 09:07:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:07:44.585090 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94af213-0b86-46ee-b57d-a437681f4c4d None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:44.603012 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac2995d-57a5-41cd-8360-0391a82b6d9c None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:44.644108 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ceef736-71b5-4955-b62c-89b1e8482aa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:44.644838 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ceef736-71b5-4955-b62c-89b1e8482aa6 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:44.645191 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ceef736-71b5-4955-b62c-89b1e8482aa6 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:44.645532 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ceef736-71b5-4955-b62c-89b1e8482aa6 None admin] PATH_INFO: `/v3/projects/99f7f068dc714797ac5d88928535f7a2` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:44.646727 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ceef736-71b5-4955-b62c-89b1e8482aa6 None admin] RBAC: Authorizing `identity:get_project(project_id=99f7f068dc714797ac5d88928535f7a2)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:44.648423 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ceef736-71b5-4955-b62c-89b1e8482aa6 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:44.650280 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 128/255] 10.222.0.16 () {56 vars in 1186 bytes} [Wed Aug 24 09:07:44 2022] GET /identity/v3/projects/99f7f068dc714797ac5d88928535f7a2 => generated 302 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:07:44.651003 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ceef736-71b5-4955-b62c-89b1e8482aa6 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:44.677810 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:44.680500 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:44.738042 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94af213-0b86-46ee-b57d-a437681f4c4d None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:44.765546 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ceef736-71b5-4955-b62c-89b1e8482aa6 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:44.779763 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc48780d-cc7b-4867-a296-7fdca9c25137 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:44.780299 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc48780d-cc7b-4867-a296-7fdca9c25137 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:44.780673 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc48780d-cc7b-4867-a296-7fdca9c25137 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:44.781142 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc48780d-cc7b-4867-a296-7fdca9c25137 None admin] PATH_INFO: `/v3/users/7ceed91937fe4c419f33b4854f0e8d6b` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:44.782495 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc48780d-cc7b-4867-a296-7fdca9c25137 None admin] RBAC: Authorizing `identity:get_user(user_id=7ceed91937fe4c419f33b4854f0e8d6b)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:44.784242 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc48780d-cc7b-4867-a296-7fdca9c25137 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:44.786206 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 128/256] 10.222.0.16 () {56 vars in 1177 bytes} [Wed Aug 24 09:07:44 2022] GET /identity/v3/users/7ceed91937fe4c419f33b4854f0e8d6b => generated 283 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:07:44.786909 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc48780d-cc7b-4867-a296-7fdca9c25137 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:44.816924 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d222ce-9e32-4251-a9be-a93f676c279d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:44.817951 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d222ce-9e32-4251-a9be-a93f676c279d None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:44.818332 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d222ce-9e32-4251-a9be-a93f676c279d None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:44.818932 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d222ce-9e32-4251-a9be-a93f676c279d None admin] PATH_INFO: `/v3/users/04d67adcc15b4c1496b767b46aaf9779` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:44.820881 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d222ce-9e32-4251-a9be-a93f676c279d None admin] RBAC: Authorizing `identity:get_user(user_id=04d67adcc15b4c1496b767b46aaf9779)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:44.823080 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d222ce-9e32-4251-a9be-a93f676c279d None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:44.825335 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 129/257] 10.222.0.16 () {56 vars in 1177 bytes} [Wed Aug 24 09:07:44 2022] GET /identity/v3/users/04d67adcc15b4c1496b767b46aaf9779 => generated 297 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:07:44.826557 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d222ce-9e32-4251-a9be-a93f676c279d None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:44.902818 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc48780d-cc7b-4867-a296-7fdca9c25137 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:44.931246 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:44.939955 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6045269-bc91-494a-ab53-f8e75a98bc14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:44.940712 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6045269-bc91-494a-ab53-f8e75a98bc14 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:44.941060 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6045269-bc91-494a-ab53-f8e75a98bc14 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:44.941399 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6045269-bc91-494a-ab53-f8e75a98bc14 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:44.951372 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:44.953552 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6045269-bc91-494a-ab53-f8e75a98bc14 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:07:44.958911 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-e6045269-bc91-494a-ab53-f8e75a98bc14 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 09:07:44.958911 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:07:44.958911 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:07:44.958911 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:07:44.958911 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:07:44.958911 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:07:44.958911 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:07:44.958911 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:07:44.958911 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:07:44.958911 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:07:44.958911 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:07:44.958911 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:07:44.958911 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:07:44.958911 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:07:44.958911 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:07:44.958911 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:07:44.960041 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:07:44.960041 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:07:44.960041 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:07:44.960041 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:07:44.960041 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:07:44.960041 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:07:44.960041 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:07:44.960041 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:07:44.960041 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:07:44.960041 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:07:44.960041 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:07:44.960041 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Aug 24 09:07:44.960041 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:07:44.960041 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:07:44.960041 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:07:44.960041 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:07:44.961041 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:07:44.961041 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:07:44.961041 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:07:44.961041 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:07:44.961041 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:07:44.961041 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:07:44.961041 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:07:44.961041 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:07:44.961041 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:07:44.961041 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:07:44.961041 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 09:07:44.961041 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Aug 24 09:07:44.965714 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d222ce-9e32-4251-a9be-a93f676c279d None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:44.967564 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 129/258] 10.222.0.16 () {56 vars in 1114 bytes} [Wed Aug 24 09:07:44 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:07:44.969570 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6045269-bc91-494a-ab53-f8e75a98bc14 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:45.008554 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5702ca-bd9f-4768-bbd3-87f85b0a36ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:45.008887 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5702ca-bd9f-4768-bbd3-87f85b0a36ab None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:45.009127 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5702ca-bd9f-4768-bbd3-87f85b0a36ab None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:45.009408 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5702ca-bd9f-4768-bbd3-87f85b0a36ab None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:45.010149 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5702ca-bd9f-4768-bbd3-87f85b0a36ab None admin] RBAC: Adding query filter params (role.id=5a55760968cc43ef86318e5bec24f4ef, scope.project.id=99f7f068dc714797ac5d88928535f7a2, user.id=6a5fc2cfb6c44b4cb49a33930d70d857) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:07:45.010457 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5702ca-bd9f-4768-bbd3-87f85b0a36ab None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:45.011962 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5702ca-bd9f-4768-bbd3-87f85b0a36ab None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:45.018492 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 130/259] 10.222.0.16 () {56 vars in 1374 bytes} [Wed Aug 24 09:07:44 2022] GET /identity/v3/role_assignments?user.id=6a5fc2cfb6c44b4cb49a33930d70d857&scope.project.id=99f7f068dc714797ac5d88928535f7a2&role.id=5a55760968cc43ef86318e5bec24f4ef => generated 259 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:07:45.019365 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5702ca-bd9f-4768-bbd3-87f85b0a36ab None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:45.082320 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6045269-bc91-494a-ab53-f8e75a98bc14 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:45.140347 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3dafc1-745d-47ab-b095-b7fbb9fb2fd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:45.140986 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3dafc1-745d-47ab-b095-b7fbb9fb2fd3 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:45.141266 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3dafc1-745d-47ab-b095-b7fbb9fb2fd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:45.141530 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3dafc1-745d-47ab-b095-b7fbb9fb2fd3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:45.158204 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5702ca-bd9f-4768-bbd3-87f85b0a36ab None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:45.205159 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf28194-a717-4963-9dbb-24b82bca0503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:45.205799 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf28194-a717-4963-9dbb-24b82bca0503 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:45.206076 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf28194-a717-4963-9dbb-24b82bca0503 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:45.206365 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf28194-a717-4963-9dbb-24b82bca0503 None admin] PATH_INFO: `/v3/projects/99f7f068dc714797ac5d88928535f7a2` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:45.212132 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf28194-a717-4963-9dbb-24b82bca0503 None admin] RBAC: Authorizing `identity:get_project(project_id=99f7f068dc714797ac5d88928535f7a2)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:45.212132 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf28194-a717-4963-9dbb-24b82bca0503 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:45.212132 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 131/260] 10.222.0.16 () {56 vars in 1186 bytes} [Wed Aug 24 09:07:45 2022] GET /identity/v3/projects/99f7f068dc714797ac5d88928535f7a2 => generated 302 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:07:45.212132 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf28194-a717-4963-9dbb-24b82bca0503 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:45.252952 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:45.257688 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:45.271455 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3dafc1-745d-47ab-b095-b7fbb9fb2fd3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:45.273403 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3dafc1-745d-47ab-b095-b7fbb9fb2fd3 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:45.333653 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf28194-a717-4963-9dbb-24b82bca0503 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:45.369804 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a424476f-b7bc-4345-9994-8ef33d8d7a8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:45.370417 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a424476f-b7bc-4345-9994-8ef33d8d7a8c None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:45.370694 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a424476f-b7bc-4345-9994-8ef33d8d7a8c None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:45.370964 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a424476f-b7bc-4345-9994-8ef33d8d7a8c None admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:45.371667 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a424476f-b7bc-4345-9994-8ef33d8d7a8c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:07:45.371974 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a424476f-b7bc-4345-9994-8ef33d8d7a8c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:45.373391 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a424476f-b7bc-4345-9994-8ef33d8d7a8c None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:45.381747 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 132/261] 10.222.0.16 () {56 vars in 1111 bytes} [Wed Aug 24 09:07:45 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:07:45.382554 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a424476f-b7bc-4345-9994-8ef33d8d7a8c None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:45.426937 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 130/262] 10.222.0.16 () {58 vars in 1282 bytes} [Wed Aug 24 09:07:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 458 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 09:07:45.428185 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3dafc1-745d-47ab-b095-b7fbb9fb2fd3 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:45.487198 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:45.500089 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a424476f-b7bc-4345-9994-8ef33d8d7a8c None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:45.536322 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2159a3-7c8c-417b-bf29-bd0ea67853c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:45.537240 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2159a3-7c8c-417b-bf29-bd0ea67853c7 None admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:45.537584 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2159a3-7c8c-417b-bf29-bd0ea67853c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:45.537930 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2159a3-7c8c-417b-bf29-bd0ea67853c7 None admin] PATH_INFO: `/v3/projects/99f7f068dc714797ac5d88928535f7a2/users/04d67adcc15b4c1496b767b46aaf9779/roles/136cdf78e00a426997dd4e37faff489b` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:45.539989 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2159a3-7c8c-417b-bf29-bd0ea67853c7 None admin] RBAC: Authorizing `identity:create_grant(project_id=99f7f068dc714797ac5d88928535f7a2, user_id=04d67adcc15b4c1496b767b46aaf9779, role_id=136cdf78e00a426997dd4e37faff489b)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:45.543616 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2159a3-7c8c-417b-bf29-bd0ea67853c7 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:45.576114 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:45.589254 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3dafc1-745d-47ab-b095-b7fbb9fb2fd3 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:45.627941 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 133/263] 10.222.0.16 () {58 vars in 1439 bytes} [Wed Aug 24 09:07:45 2022] PUT /identity/v3/projects/99f7f068dc714797ac5d88928535f7a2/users/04d67adcc15b4c1496b767b46aaf9779/roles/136cdf78e00a426997dd4e37faff489b => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:07:45.628907 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2159a3-7c8c-417b-bf29-bd0ea67853c7 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:45.647948 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62dd4c4-7da4-4c53-9ee5-b8c1dfc5cf4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:45.648874 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62dd4c4-7da4-4c53-9ee5-b8c1dfc5cf4b None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:45.649155 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62dd4c4-7da4-4c53-9ee5-b8c1dfc5cf4b None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:45.651455 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62dd4c4-7da4-4c53-9ee5-b8c1dfc5cf4b None admin] PATH_INFO: `/v3/users/7ceed91937fe4c419f33b4854f0e8d6b` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:45.653702 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62dd4c4-7da4-4c53-9ee5-b8c1dfc5cf4b None admin] RBAC: Authorizing `identity:get_user(user_id=7ceed91937fe4c419f33b4854f0e8d6b)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:45.662253 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62dd4c4-7da4-4c53-9ee5-b8c1dfc5cf4b None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:45.667443 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 131/264] 10.222.0.16 () {56 vars in 1177 bytes} [Wed Aug 24 09:07:45 2022] GET /identity/v3/users/7ceed91937fe4c419f33b4854f0e8d6b => generated 283 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:07:45.673085 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62dd4c4-7da4-4c53-9ee5-b8c1dfc5cf4b None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:45.816892 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2159a3-7c8c-417b-bf29-bd0ea67853c7 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:45.865429 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3cf349-043f-4052-9c02-6d743ac9dd84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:45.866086 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3cf349-043f-4052-9c02-6d743ac9dd84 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:45.866375 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3cf349-043f-4052-9c02-6d743ac9dd84 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:45.866646 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3cf349-043f-4052-9c02-6d743ac9dd84 None admin] PATH_INFO: `/v3/projects/99f7f068dc714797ac5d88928535f7a2` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:45.867955 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3cf349-043f-4052-9c02-6d743ac9dd84 None admin] RBAC: Authorizing `identity:get_project(project_id=99f7f068dc714797ac5d88928535f7a2)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:45.869417 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3cf349-043f-4052-9c02-6d743ac9dd84 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:45.877469 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 134/265] 10.222.0.16 () {56 vars in 1186 bytes} [Wed Aug 24 09:07:45 2022] GET /identity/v3/projects/99f7f068dc714797ac5d88928535f7a2 => generated 302 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:07:45.886112 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3cf349-043f-4052-9c02-6d743ac9dd84 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:45.888897 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62dd4c4-7da4-4c53-9ee5-b8c1dfc5cf4b None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:45.915972 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:45.967688 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfb1bcd-4c19-47fd-9c9f-8421b5981708 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:45.968319 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfb1bcd-4c19-47fd-9c9f-8421b5981708 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:45.968650 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfb1bcd-4c19-47fd-9c9f-8421b5981708 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:45.968925 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfb1bcd-4c19-47fd-9c9f-8421b5981708 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:46.027963 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:46.087296 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3cf349-043f-4052-9c02-6d743ac9dd84 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:46.088435 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfb1bcd-4c19-47fd-9c9f-8421b5981708 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:46.090924 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfb1bcd-4c19-47fd-9c9f-8421b5981708 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:46.124238 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cd3e4e-b2ee-4a4f-9afb-66a8c3ee789f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:46.124849 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cd3e4e-b2ee-4a4f-9afb-66a8c3ee789f None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:46.125129 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cd3e4e-b2ee-4a4f-9afb-66a8c3ee789f None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:46.125501 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cd3e4e-b2ee-4a4f-9afb-66a8c3ee789f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:46.126336 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cd3e4e-b2ee-4a4f-9afb-66a8c3ee789f None admin] RBAC: Adding query filter params (role.id=15f634d4a2d247ddab90c458e5e1c141, scope.project.id=99f7f068dc714797ac5d88928535f7a2, user.id=7ceed91937fe4c419f33b4854f0e8d6b) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:07:46.126648 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cd3e4e-b2ee-4a4f-9afb-66a8c3ee789f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:46.128114 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cd3e4e-b2ee-4a4f-9afb-66a8c3ee789f None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:46.134639 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 135/266] 10.222.0.16 () {56 vars in 1374 bytes} [Wed Aug 24 09:07:45 2022] GET /identity/v3/role_assignments?user.id=7ceed91937fe4c419f33b4854f0e8d6b&scope.project.id=99f7f068dc714797ac5d88928535f7a2&role.id=15f634d4a2d247ddab90c458e5e1c141 => generated 608 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:07:46.135860 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0165920e-39d7-457a-9541-65b2968785b4 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:46.136144 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0165920e-39d7-457a-9541-65b2968785b4 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:46.136416 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0165920e-39d7-457a-9541-65b2968785b4 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:46.137627 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 136/267] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:07:46 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:07:46.143143 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717fd99f-3b87-4f59-b571-fb52a79d3ae7 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:46.143426 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717fd99f-3b87-4f59-b571-fb52a79d3ae7 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:46.143887 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717fd99f-3b87-4f59-b571-fb52a79d3ae7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:46.167976 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-717fd99f-3b87-4f59-b571-fb52a79d3ae7 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:07:46.170213 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-717fd99f-3b87-4f59-b571-fb52a79d3ae7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:07:46.271856 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 132/268] 10.222.0.16 () {58 vars in 1282 bytes} [Wed Aug 24 09:07:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 599 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 09:07:46.273572 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7dc60a-2dc4-4744-9a13-2550e013a2cd None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:46.274870 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7dc60a-2dc4-4744-9a13-2550e013a2cd None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:46.275337 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7dc60a-2dc4-4744-9a13-2550e013a2cd None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:46.277225 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 133/269] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:07:46 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:07:46.278318 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7dc60a-2dc4-4744-9a13-2550e013a2cd None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:46.347347 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 137/270] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:07:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 09:07:46.348921 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70551711-1710-429a-895e-7e1f9af88205 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:46.349373 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70551711-1710-429a-895e-7e1f9af88205 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:46.349659 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70551711-1710-429a-895e-7e1f9af88205 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:46.369082 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-70551711-1710-429a-895e-7e1f9af88205 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:07:46.370377 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-70551711-1710-429a-895e-7e1f9af88205 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:07:46.373940 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:46.405643 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:46.450719 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 138/271] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:07:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:07:46.503780 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7dc60a-2dc4-4744-9a13-2550e013a2cd None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:46.549171 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d482c352-e57f-4faf-9f32-2ecd5783dbe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:46.549805 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d482c352-e57f-4faf-9f32-2ecd5783dbe7 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:46.550082 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d482c352-e57f-4faf-9f32-2ecd5783dbe7 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:46.550352 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d482c352-e57f-4faf-9f32-2ecd5783dbe7 None admin] PATH_INFO: `/v3/projects/99f7f068dc714797ac5d88928535f7a2` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:46.554534 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d482c352-e57f-4faf-9f32-2ecd5783dbe7 None admin] RBAC: Authorizing `identity:get_project(project_id=99f7f068dc714797ac5d88928535f7a2)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:46.556010 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d482c352-e57f-4faf-9f32-2ecd5783dbe7 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:46.557965 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 134/272] 10.222.0.16 () {56 vars in 1186 bytes} [Wed Aug 24 09:07:46 2022] GET /identity/v3/projects/99f7f068dc714797ac5d88928535f7a2 => generated 302 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:07:46.564409 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70551711-1710-429a-895e-7e1f9af88205 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:46.639611 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d482c352-e57f-4faf-9f32-2ecd5783dbe7 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:46.692570 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70551711-1710-429a-895e-7e1f9af88205 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:46.760401 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112dbfa7-6660-478e-be87-f669e0739b0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:46.761024 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112dbfa7-6660-478e-be87-f669e0739b0c None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:46.761296 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112dbfa7-6660-478e-be87-f669e0739b0c None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:46.761561 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112dbfa7-6660-478e-be87-f669e0739b0c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:46.762298 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112dbfa7-6660-478e-be87-f669e0739b0c None admin] RBAC: Adding query filter params (role.id=ed6466d17fb7403c8491bb7f1ef95fc1, scope.project.id=99f7f068dc714797ac5d88928535f7a2, user.id=7ceed91937fe4c419f33b4854f0e8d6b) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:07:46.762599 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112dbfa7-6660-478e-be87-f669e0739b0c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:46.765981 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112dbfa7-6660-478e-be87-f669e0739b0c None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:46.770800 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:46.776577 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 139/273] 10.222.0.16 () {56 vars in 1374 bytes} [Wed Aug 24 09:07:46 2022] GET /identity/v3/role_assignments?user.id=7ceed91937fe4c419f33b4854f0e8d6b&scope.project.id=99f7f068dc714797ac5d88928535f7a2&role.id=ed6466d17fb7403c8491bb7f1ef95fc1 => generated 608 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:07:46.777251 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112dbfa7-6660-478e-be87-f669e0739b0c None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:46.867881 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d482c352-e57f-4faf-9f32-2ecd5783dbe7 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:46.897357 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:46.909868 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862991db-8a44-4a92-8ab3-d63b670c6933 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:46.910515 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862991db-8a44-4a92-8ab3-d63b670c6933 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:46.910788 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862991db-8a44-4a92-8ab3-d63b670c6933 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:46.911058 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862991db-8a44-4a92-8ab3-d63b670c6933 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:46.912781 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862991db-8a44-4a92-8ab3-d63b670c6933 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:46.914544 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862991db-8a44-4a92-8ab3-d63b670c6933 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:46.916468 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 135/274] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 24 09:07:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:07:46.922765 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862991db-8a44-4a92-8ab3-d63b670c6933 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:46.978929 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112dbfa7-6660-478e-be87-f669e0739b0c None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:47.027703 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12d9dbb-278f-41d0-9d67-6dfba0c047eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:47.028730 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12d9dbb-278f-41d0-9d67-6dfba0c047eb None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:47.029238 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12d9dbb-278f-41d0-9d67-6dfba0c047eb None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:47.029767 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12d9dbb-278f-41d0-9d67-6dfba0c047eb None admin] PATH_INFO: `/v3/roles/member` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:47.041675 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12d9dbb-278f-41d0-9d67-6dfba0c047eb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:07:47.047740 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-c12d9dbb-278f-41d0-9d67-6dfba0c047eb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 09:07:47.047740 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:07:47.047740 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:07:47.047740 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:07:47.047740 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:07:47.047740 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:07:47.047740 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:07:47.047740 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:07:47.047740 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:07:47.047740 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:07:47.047740 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:07:47.047740 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:07:47.047740 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:07:47.047740 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:07:47.047740 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:07:47.047740 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:07:47.048677 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:07:47.048677 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:07:47.048677 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:07:47.048677 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:07:47.048677 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:07:47.048677 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:07:47.048677 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:07:47.048677 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:07:47.048677 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:07:47.048677 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:07:47.048677 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:07:47.048677 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 24 09:07:47.048677 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:07:47.048677 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:07:47.048677 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:07:47.048677 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:07:47.049475 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:07:47.049475 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:07:47.049475 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:07:47.049475 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:07:47.049475 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:07:47.049475 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:07:47.049475 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:07:47.049475 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:07:47.049475 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:07:47.049475 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:07:47.049475 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 09:07:47.049475 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 24 09:07:47.052333 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 140/275] 10.222.0.16 () {56 vars in 1099 bytes} [Wed Aug 24 09:07:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:07:47.057154 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862991db-8a44-4a92-8ab3-d63b670c6933 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:47.057873 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12d9dbb-278f-41d0-9d67-6dfba0c047eb None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:47.108691 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1854138-e87c-4932-aa79-297e7cb106d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:47.109661 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1854138-e87c-4932-aa79-297e7cb106d7 None admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:47.110245 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1854138-e87c-4932-aa79-297e7cb106d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:47.110720 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1854138-e87c-4932-aa79-297e7cb106d7 None admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:47.111572 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1854138-e87c-4932-aa79-297e7cb106d7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:47.113727 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1854138-e87c-4932-aa79-297e7cb106d7 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:47.146978 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:47.194886 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12d9dbb-278f-41d0-9d67-6dfba0c047eb None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:47.219730 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 136/276] 10.222.0.16 () {60 vars in 1132 bytes} [Wed Aug 24 09:07:46 2022] POST /identity/v3/users => generated 293 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 09:07:47.235465 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0baa921b-9e35-43bc-b969-a9509c4fbb20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:47.236121 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baa921b-9e35-43bc-b969-a9509c4fbb20 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:47.236400 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baa921b-9e35-43bc-b969-a9509c4fbb20 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:47.236673 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baa921b-9e35-43bc-b969-a9509c4fbb20 None admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:47.237353 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0baa921b-9e35-43bc-b969-a9509c4fbb20 None admin] RBAC: Adding query filter params (name=member) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:07:47.237656 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0baa921b-9e35-43bc-b969-a9509c4fbb20 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:47.239061 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0baa921b-9e35-43bc-b969-a9509c4fbb20 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:47.249545 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 141/277] 10.222.0.16 () {56 vars in 1101 bytes} [Wed Aug 24 09:07:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:07:47.255355 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1854138-e87c-4932-aa79-297e7cb106d7 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:47.317387 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:47.379107 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1854138-e87c-4932-aa79-297e7cb106d7 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:47.415127 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0e0e21-537c-4000-9147-43effc54e574 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:47.415807 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0e0e21-537c-4000-9147-43effc54e574 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:47.416094 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0e0e21-537c-4000-9147-43effc54e574 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:47.416373 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0e0e21-537c-4000-9147-43effc54e574 None admin] PATH_INFO: `/v3/users/db38a9feea854ecfbb16c49e37d7c6d4` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:47.432732 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0e0e21-537c-4000-9147-43effc54e574 None admin] RBAC: Authorizing `identity:get_user(user_id=db38a9feea854ecfbb16c49e37d7c6d4)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:47.434202 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0e0e21-537c-4000-9147-43effc54e574 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:47.436865 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 137/278] 10.222.0.16 () {56 vars in 1177 bytes} [Wed Aug 24 09:07:47 2022] GET /identity/v3/users/db38a9feea854ecfbb16c49e37d7c6d4 => generated 293 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:07:47.442586 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0baa921b-9e35-43bc-b969-a9509c4fbb20 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:47.555952 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0baa921b-9e35-43bc-b969-a9509c4fbb20 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:47.583483 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:47.595664 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548940db-7de1-40f9-aa7c-a81bb0dc831c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:47.596278 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548940db-7de1-40f9-aa7c-a81bb0dc831c None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:47.596561 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548940db-7de1-40f9-aa7c-a81bb0dc831c None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:47.596833 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548940db-7de1-40f9-aa7c-a81bb0dc831c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:47.597622 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548940db-7de1-40f9-aa7c-a81bb0dc831c None admin] RBAC: Adding query filter params (role.id=136cdf78e00a426997dd4e37faff489b, user.id=db38a9feea854ecfbb16c49e37d7c6d4, scope.system=all) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:07:47.597882 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548940db-7de1-40f9-aa7c-a81bb0dc831c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:47.599368 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548940db-7de1-40f9-aa7c-a81bb0dc831c None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:47.605232 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 142/279] 10.222.0.16 () {56 vars in 1308 bytes} [Wed Aug 24 09:07:47 2022] GET /identity/v3/role_assignments?user.id=db38a9feea854ecfbb16c49e37d7c6d4&scope.system=all&role.id=136cdf78e00a426997dd4e37faff489b => generated 226 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:07:47.633713 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7999e332-8cd4-47d8-b096-11979069ef34 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:47.633998 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7999e332-8cd4-47d8-b096-11979069ef34 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:47.634277 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7999e332-8cd4-47d8-b096-11979069ef34 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:47.635917 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 138/280] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:07:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:07:47.642439 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c958af1-0eb6-4804-9378-692aecc4d77f None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:47.642806 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c958af1-0eb6-4804-9378-692aecc4d77f None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:47.643000 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c958af1-0eb6-4804-9378-692aecc4d77f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:47.666547 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-7c958af1-0eb6-4804-9378-692aecc4d77f None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:07:47.667827 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-7c958af1-0eb6-4804-9378-692aecc4d77f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:07:47.747861 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 143/281] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:07:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:07:48.020337 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7999e332-8cd4-47d8-b096-11979069ef34 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:48.129268 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7999e332-8cd4-47d8-b096-11979069ef34 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:48.137371 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:48.165083 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d60c98-82a9-4063-babd-e0db8e5c64c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:48.165724 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d60c98-82a9-4063-babd-e0db8e5c64c6 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:48.166003 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d60c98-82a9-4063-babd-e0db8e5c64c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:48.166277 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d60c98-82a9-4063-babd-e0db8e5c64c6 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:48.177802 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d60c98-82a9-4063-babd-e0db8e5c64c6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:07:48.182927 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-14d60c98-82a9-4063-babd-e0db8e5c64c6 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 09:07:48.182927 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:07:48.182927 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:07:48.182927 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:07:48.182927 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:07:48.182927 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:07:48.182927 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:07:48.182927 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:07:48.182927 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:07:48.182927 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:07:48.182927 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:07:48.182927 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:07:48.182927 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:07:48.182927 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:07:48.182927 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:07:48.182927 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:07:48.183854 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:07:48.183854 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:07:48.183854 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:07:48.183854 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:07:48.183854 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:07:48.183854 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:07:48.183854 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:07:48.183854 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:07:48.183854 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:07:48.183854 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:07:48.183854 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:07:48.183854 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Aug 24 09:07:48.183854 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:07:48.183854 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:07:48.183854 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:07:48.183854 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:07:48.184640 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:07:48.184640 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:07:48.184640 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:07:48.184640 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:07:48.184640 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:07:48.184640 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:07:48.184640 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:07:48.184640 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:07:48.184640 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:07:48.184640 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:07:48.184640 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 09:07:48.184640 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Aug 24 09:07:48.189613 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 139/282] 10.222.0.16 () {56 vars in 1099 bytes} [Wed Aug 24 09:07:48 2022] GET /identity/v3/roles/reader => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:07:48.195234 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c958af1-0eb6-4804-9378-692aecc4d77f None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:48.241478 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ba2614-a506-423f-afa2-dce687f41454 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:48.241797 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ba2614-a506-423f-afa2-dce687f41454 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:48.242023 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ba2614-a506-423f-afa2-dce687f41454 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:48.243507 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 140/283] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:07:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 09:07:48.249277 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69250d8d-e7dd-4136-86bf-326d37d0e855 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:48.249564 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69250d8d-e7dd-4136-86bf-326d37d0e855 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:48.249842 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69250d8d-e7dd-4136-86bf-326d37d0e855 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:48.269158 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-69250d8d-e7dd-4136-86bf-326d37d0e855 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:07:48.270544 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-69250d8d-e7dd-4136-86bf-326d37d0e855 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:07:48.345411 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 141/284] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:07:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Aug 24 09:07:48.369720 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c958af1-0eb6-4804-9378-692aecc4d77f None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:48.401466 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:48.431516 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89ceae6-7928-4b2c-ab5d-8e006647d98f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:48.432165 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89ceae6-7928-4b2c-ab5d-8e006647d98f None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:48.432444 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89ceae6-7928-4b2c-ab5d-8e006647d98f None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:48.432716 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89ceae6-7928-4b2c-ab5d-8e006647d98f None admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:48.433387 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89ceae6-7928-4b2c-ab5d-8e006647d98f None admin] RBAC: Adding query filter params (name=reader) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:07:48.433695 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89ceae6-7928-4b2c-ab5d-8e006647d98f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:48.435085 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89ceae6-7928-4b2c-ab5d-8e006647d98f None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:48.449152 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 144/285] 10.222.0.16 () {56 vars in 1101 bytes} [Wed Aug 24 09:07:48 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:07:48.457615 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69250d8d-e7dd-4136-86bf-326d37d0e855 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:48.566825 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:48.573390 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69250d8d-e7dd-4136-86bf-326d37d0e855 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:48.614133 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b04acd-39d8-40e8-8732-f32d37383435 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:48.615255 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b04acd-39d8-40e8-8732-f32d37383435 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:48.615766 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b04acd-39d8-40e8-8732-f32d37383435 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:48.616245 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b04acd-39d8-40e8-8732-f32d37383435 None admin] PATH_INFO: `/v3/users/6a5fc2cfb6c44b4cb49a33930d70d857` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:48.620245 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b04acd-39d8-40e8-8732-f32d37383435 None admin] RBAC: Authorizing `identity:get_user(user_id=6a5fc2cfb6c44b4cb49a33930d70d857)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:48.621942 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b04acd-39d8-40e8-8732-f32d37383435 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:48.624146 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 142/286] 10.222.0.16 () {56 vars in 1177 bytes} [Wed Aug 24 09:07:48 2022] GET /identity/v3/users/6a5fc2cfb6c44b4cb49a33930d70d857 => generated 297 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:07:48.629986 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89ceae6-7928-4b2c-ab5d-8e006647d98f None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:48.759176 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b04acd-39d8-40e8-8732-f32d37383435 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:48.798321 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89ceae6-7928-4b2c-ab5d-8e006647d98f None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:48.861725 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8744e80-526b-4223-9a81-73fe2cb552b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:48.862282 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8744e80-526b-4223-9a81-73fe2cb552b8 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:48.862562 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8744e80-526b-4223-9a81-73fe2cb552b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:48.862832 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8744e80-526b-4223-9a81-73fe2cb552b8 None admin] PATH_INFO: `/v3/projects/99f7f068dc714797ac5d88928535f7a2` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:48.868276 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8744e80-526b-4223-9a81-73fe2cb552b8 None admin] RBAC: Authorizing `identity:get_project(project_id=99f7f068dc714797ac5d88928535f7a2)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:48.869732 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8744e80-526b-4223-9a81-73fe2cb552b8 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:48.875607 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 145/287] 10.222.0.16 () {56 vars in 1186 bytes} [Wed Aug 24 09:07:48 2022] GET /identity/v3/projects/99f7f068dc714797ac5d88928535f7a2 => generated 302 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:07:48.879417 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b04acd-39d8-40e8-8732-f32d37383435 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:48.884556 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8744e80-526b-4223-9a81-73fe2cb552b8 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:48.918605 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42271ec7-0447-4047-bcee-c59352a762c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:48.920289 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42271ec7-0447-4047-bcee-c59352a762c9 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:48.920776 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42271ec7-0447-4047-bcee-c59352a762c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:48.921266 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42271ec7-0447-4047-bcee-c59352a762c9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:48.933065 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42271ec7-0447-4047-bcee-c59352a762c9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:07:48.936074 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:48.940851 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-42271ec7-0447-4047-bcee-c59352a762c9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 09:07:48.940851 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:07:48.940851 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:07:48.940851 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:07:48.940851 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:07:48.940851 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:07:48.940851 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:07:48.940851 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:07:48.940851 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:07:48.940851 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:07:48.940851 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:07:48.940851 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:07:48.940851 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:07:48.940851 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:07:48.940851 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:07:48.940851 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:07:48.941748 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:07:48.941748 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:07:48.941748 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:07:48.941748 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:07:48.941748 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:07:48.941748 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:07:48.941748 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:07:48.941748 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:07:48.941748 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:07:48.941748 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:07:48.941748 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:07:48.941748 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Aug 24 09:07:48.941748 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:07:48.941748 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:07:48.941748 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:07:48.941748 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:07:48.942558 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:07:48.942558 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:07:48.942558 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:07:48.942558 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:07:48.942558 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:07:48.942558 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:07:48.942558 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:07:48.942558 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:07:48.942558 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:07:48.942558 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:07:48.942558 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 09:07:48.942558 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Aug 24 09:07:48.949686 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 143/288] 10.222.0.16 () {56 vars in 1099 bytes} [Wed Aug 24 09:07:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 09:07:48.951956 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42271ec7-0447-4047-bcee-c59352a762c9 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:48.981090 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:49.071327 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42271ec7-0447-4047-bcee-c59352a762c9 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:49.112847 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67f5e27-a290-4fef-a062-361ca2134618 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:49.113811 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67f5e27-a290-4fef-a062-361ca2134618 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:49.114303 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67f5e27-a290-4fef-a062-361ca2134618 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:49.114778 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67f5e27-a290-4fef-a062-361ca2134618 None admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:49.115679 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67f5e27-a290-4fef-a062-361ca2134618 None admin] RBAC: Adding query filter params (name=member) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:07:49.116205 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67f5e27-a290-4fef-a062-361ca2134618 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:49.118567 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67f5e27-a290-4fef-a062-361ca2134618 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:49.124074 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8744e80-526b-4223-9a81-73fe2cb552b8 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:49.137748 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 144/289] 10.222.0.16 () {56 vars in 1101 bytes} [Wed Aug 24 09:07:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:07:49.147839 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67f5e27-a290-4fef-a062-361ca2134618 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:49.202210 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd21f04d-2a62-4f07-b993-68675c91640f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:49.203453 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd21f04d-2a62-4f07-b993-68675c91640f None admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:49.203983 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd21f04d-2a62-4f07-b993-68675c91640f None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:49.204463 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd21f04d-2a62-4f07-b993-68675c91640f None admin] PATH_INFO: `/v3/projects/99f7f068dc714797ac5d88928535f7a2/users/6a5fc2cfb6c44b4cb49a33930d70d857/roles/5a55760968cc43ef86318e5bec24f4ef` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:49.206878 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd21f04d-2a62-4f07-b993-68675c91640f None admin] RBAC: Authorizing `identity:create_grant(project_id=99f7f068dc714797ac5d88928535f7a2, user_id=6a5fc2cfb6c44b4cb49a33930d70d857, role_id=5a55760968cc43ef86318e5bec24f4ef)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:49.208613 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd21f04d-2a62-4f07-b993-68675c91640f None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:49.275609 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 146/290] 10.222.0.16 () {58 vars in 1439 bytes} [Wed Aug 24 09:07:48 2022] PUT /identity/v3/projects/99f7f068dc714797ac5d88928535f7a2/users/6a5fc2cfb6c44b4cb49a33930d70d857/roles/5a55760968cc43ef86318e5bec24f4ef => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:07:49.312077 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:49.330923 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67f5e27-a290-4fef-a062-361ca2134618 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:49.372970 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ab582a-9d07-4f83-8cba-b158ec2e690f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:49.373552 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ab582a-9d07-4f83-8cba-b158ec2e690f None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:49.373831 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ab582a-9d07-4f83-8cba-b158ec2e690f None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:49.374105 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ab582a-9d07-4f83-8cba-b158ec2e690f None admin] PATH_INFO: `/v3/users/04d67adcc15b4c1496b767b46aaf9779` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:49.375343 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ab582a-9d07-4f83-8cba-b158ec2e690f None admin] RBAC: Authorizing `identity:get_user(user_id=04d67adcc15b4c1496b767b46aaf9779)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:49.376987 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ab582a-9d07-4f83-8cba-b158ec2e690f None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:49.378734 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 145/291] 10.222.0.16 () {56 vars in 1177 bytes} [Wed Aug 24 09:07:49 2022] GET /identity/v3/users/04d67adcc15b4c1496b767b46aaf9779 => generated 297 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:07:49.385738 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd21f04d-2a62-4f07-b993-68675c91640f None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:49.501276 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd21f04d-2a62-4f07-b993-68675c91640f None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:49.508663 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:49.536583 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5f5d0d-5fb7-4ef5-8b9b-07df2083e7ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:49.537192 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5f5d0d-5fb7-4ef5-8b9b-07df2083e7ec None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:49.537468 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5f5d0d-5fb7-4ef5-8b9b-07df2083e7ec None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:49.537737 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5f5d0d-5fb7-4ef5-8b9b-07df2083e7ec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:49.589508 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093c3903-5307-4da5-b503-bd70cb6b4bc4 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:49.590289 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093c3903-5307-4da5-b503-bd70cb6b4bc4 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:49.590834 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093c3903-5307-4da5-b503-bd70cb6b4bc4 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:49.592785 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 146/292] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:07:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:07:49.600327 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ed628b-da23-42ed-b8de-50e246664d7f None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:49.600616 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ed628b-da23-42ed-b8de-50e246664d7f None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:49.600894 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ed628b-da23-42ed-b8de-50e246664d7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:49.632746 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5f5d0d-5fb7-4ef5-8b9b-07df2083e7ec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:49.634150 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5f5d0d-5fb7-4ef5-8b9b-07df2083e7ec None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:49.636927 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-45ed628b-da23-42ed-b8de-50e246664d7f None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:07:49.638165 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-45ed628b-da23-42ed-b8de-50e246664d7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:07:49.749005 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 147/293] 10.222.0.16 () {58 vars in 1282 bytes} [Wed Aug 24 09:07:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 09:07:49.754484 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5f5d0d-5fb7-4ef5-8b9b-07df2083e7ec None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:49.762594 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:49.841781 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 147/294] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:07:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:07:49.857939 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5f5d0d-5fb7-4ef5-8b9b-07df2083e7ec None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:49.892297 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9590f70-7c02-4c8c-b030-8a607e876749 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:49.892971 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9590f70-7c02-4c8c-b030-8a607e876749 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:49.893317 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9590f70-7c02-4c8c-b030-8a607e876749 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:49.893665 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9590f70-7c02-4c8c-b030-8a607e876749 None admin] PATH_INFO: `/v3/projects/99f7f068dc714797ac5d88928535f7a2` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:49.894850 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9590f70-7c02-4c8c-b030-8a607e876749 None admin] RBAC: Authorizing `identity:get_project(project_id=99f7f068dc714797ac5d88928535f7a2)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:49.896396 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9590f70-7c02-4c8c-b030-8a607e876749 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:49.903166 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 148/295] 10.222.0.16 () {56 vars in 1186 bytes} [Wed Aug 24 09:07:49 2022] GET /identity/v3/projects/99f7f068dc714797ac5d88928535f7a2 => generated 302 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:07:49.904464 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ed628b-da23-42ed-b8de-50e246664d7f None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:49.994064 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:50.002040 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8ab87f-e48e-4b37-a6d7-d21a29d3dab6 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:50.002583 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8ab87f-e48e-4b37-a6d7-d21a29d3dab6 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:50.003050 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8ab87f-e48e-4b37-a6d7-d21a29d3dab6 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:50.004875 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 149/296] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:07:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:07:50.009483 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea47707c-5c50-4879-9bc7-41f9d2ecdd2c None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:50.009981 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea47707c-5c50-4879-9bc7-41f9d2ecdd2c None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:50.010407 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea47707c-5c50-4879-9bc7-41f9d2ecdd2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:50.026703 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ed628b-da23-42ed-b8de-50e246664d7f None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:50.040513 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-ea47707c-5c50-4879-9bc7-41f9d2ecdd2c None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:07:50.042755 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-ea47707c-5c50-4879-9bc7-41f9d2ecdd2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:07:50.089823 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2caadb5-3f1a-436e-b96e-ae5307779038 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:50.090525 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2caadb5-3f1a-436e-b96e-ae5307779038 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:50.091700 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2caadb5-3f1a-436e-b96e-ae5307779038 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:50.091825 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2caadb5-3f1a-436e-b96e-ae5307779038 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:50.092589 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2caadb5-3f1a-436e-b96e-ae5307779038 None admin] RBAC: Adding query filter params (role.id=136cdf78e00a426997dd4e37faff489b, scope.project.id=99f7f068dc714797ac5d88928535f7a2, user.id=04d67adcc15b4c1496b767b46aaf9779) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:07:50.092899 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2caadb5-3f1a-436e-b96e-ae5307779038 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:50.094443 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2caadb5-3f1a-436e-b96e-ae5307779038 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:50.100997 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 148/297] 10.222.0.16 () {56 vars in 1374 bytes} [Wed Aug 24 09:07:49 2022] GET /identity/v3/role_assignments?user.id=04d67adcc15b4c1496b767b46aaf9779&scope.project.id=99f7f068dc714797ac5d88928535f7a2&role.id=136cdf78e00a426997dd4e37faff489b => generated 608 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:07:50.115318 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a244c1-b764-4191-8f3d-2ced9a14c478 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:50.115664 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a244c1-b764-4191-8f3d-2ced9a14c478 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:50.115890 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a244c1-b764-4191-8f3d-2ced9a14c478 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:50.117556 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 149/298] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:07:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:07:50.123167 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6f2947-6995-416a-8473-c1c53baa6428 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:50.123460 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6f2947-6995-416a-8473-c1c53baa6428 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:50.123745 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6f2947-6995-416a-8473-c1c53baa6428 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:50.129068 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 150/299] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:07:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 09:07:50.158011 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-1c6f2947-6995-416a-8473-c1c53baa6428 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:07:50.158011 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-1c6f2947-6995-416a-8473-c1c53baa6428 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:07:50.206947 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea47707c-5c50-4879-9bc7-41f9d2ecdd2c None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:50.211412 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:50.274556 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 150/300] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:07:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:07:50.343796 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea47707c-5c50-4879-9bc7-41f9d2ecdd2c None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:50.384218 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8c6e16-f9b0-4baa-b0d8-a355865da9df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:50.384859 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8c6e16-f9b0-4baa-b0d8-a355865da9df None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:50.385140 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8c6e16-f9b0-4baa-b0d8-a355865da9df None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:50.385415 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8c6e16-f9b0-4baa-b0d8-a355865da9df None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:50.396867 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8c6e16-f9b0-4baa-b0d8-a355865da9df None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:07:50.401733 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-dc8c6e16-f9b0-4baa-b0d8-a355865da9df None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 09:07:50.401733 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:07:50.401733 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:07:50.401733 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:07:50.401733 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:07:50.401733 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:07:50.401733 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:07:50.401733 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:07:50.401733 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:07:50.401733 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:07:50.401733 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:07:50.401733 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:07:50.401733 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:07:50.401733 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:07:50.401733 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:07:50.401733 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:07:50.402594 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:07:50.402594 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:07:50.402594 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:07:50.402594 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:07:50.402594 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:07:50.402594 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:07:50.402594 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:07:50.402594 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:07:50.402594 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:07:50.402594 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:07:50.402594 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:07:50.402594 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 24 09:07:50.402594 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:07:50.402594 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:07:50.402594 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:07:50.402594 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:07:50.404278 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:07:50.404278 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:07:50.404278 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:07:50.404278 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:07:50.404278 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:07:50.404278 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:07:50.404278 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:07:50.404278 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:07:50.404278 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:07:50.404278 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:07:50.404278 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 09:07:50.404278 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 24 09:07:50.404278 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 151/301] 10.222.0.16 () {56 vars in 1099 bytes} [Wed Aug 24 09:07:50 2022] GET /identity/v3/roles/reader => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 09:07:50.409642 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6f2947-6995-416a-8473-c1c53baa6428 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:50.429503 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8c6e16-f9b0-4baa-b0d8-a355865da9df None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:50.513231 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:50.521161 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:50.553265 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6f2947-6995-416a-8473-c1c53baa6428 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:50.557525 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8c6e16-f9b0-4baa-b0d8-a355865da9df None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:50.598479 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27dafe7-faf5-4622-8197-789fb69cff79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:50.599113 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27dafe7-faf5-4622-8197-789fb69cff79 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:50.599390 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27dafe7-faf5-4622-8197-789fb69cff79 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:50.599685 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27dafe7-faf5-4622-8197-789fb69cff79 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:50.600297 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee99dc1a-08d2-4bae-965c-1fd4d67cddd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:50.600910 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee99dc1a-08d2-4bae-965c-1fd4d67cddd6 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:50.601189 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee99dc1a-08d2-4bae-965c-1fd4d67cddd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:50.601459 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee99dc1a-08d2-4bae-965c-1fd4d67cddd6 None admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:50.602229 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee99dc1a-08d2-4bae-965c-1fd4d67cddd6 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:07:50.602714 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee99dc1a-08d2-4bae-965c-1fd4d67cddd6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:50.604120 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee99dc1a-08d2-4bae-965c-1fd4d67cddd6 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:50.610861 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27dafe7-faf5-4622-8197-789fb69cff79 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:07:50.614576 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 151/302] 10.222.0.16 () {56 vars in 1101 bytes} [Wed Aug 24 09:07:50 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:07:50.619203 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee99dc1a-08d2-4bae-965c-1fd4d67cddd6 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:50.619293 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-b27dafe7-faf5-4622-8197-789fb69cff79 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 09:07:50.619293 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:07:50.619293 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:07:50.619293 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:07:50.619293 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:07:50.619293 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:07:50.619293 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:07:50.619293 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:07:50.619293 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:07:50.619293 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:07:50.619293 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:07:50.619293 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:07:50.619293 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:07:50.619293 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:07:50.619293 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:07:50.619293 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:07:50.620342 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:07:50.620342 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:07:50.620342 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:07:50.620342 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:07:50.620342 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:07:50.620342 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:07:50.620342 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:07:50.620342 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:07:50.620342 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:07:50.620342 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:07:50.620342 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:07:50.620342 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 24 09:07:50.620342 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:07:50.620342 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:07:50.620342 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:07:50.620342 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:07:50.621149 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:07:50.621149 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:07:50.621149 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:07:50.621149 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:07:50.621149 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:07:50.621149 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:07:50.621149 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:07:50.621149 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:07:50.621149 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:07:50.621149 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:07:50.621149 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 09:07:50.621149 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 24 09:07:50.621149 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 152/303] 10.222.0.16 () {56 vars in 1099 bytes} [Wed Aug 24 09:07:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 09:07:50.622142 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27dafe7-faf5-4622-8197-789fb69cff79 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:50.732559 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee99dc1a-08d2-4bae-965c-1fd4d67cddd6 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:50.733188 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27dafe7-faf5-4622-8197-789fb69cff79 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:50.764896 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:50.770607 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e553a60-1f1c-4047-a471-38863b1bb9ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:50.771239 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e553a60-1f1c-4047-a471-38863b1bb9ed None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:50.771515 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e553a60-1f1c-4047-a471-38863b1bb9ed None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:50.771814 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e553a60-1f1c-4047-a471-38863b1bb9ed None admin] PATH_INFO: `/v3/domains/default` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:50.772908 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e553a60-1f1c-4047-a471-38863b1bb9ed None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:50.774346 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e553a60-1f1c-4047-a471-38863b1bb9ed None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:50.775926 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 152/304] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 24 09:07:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:07:50.776023 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20df8ed1-e510-4a87-b4eb-5bdfadedafed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:50.786134 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20df8ed1-e510-4a87-b4eb-5bdfadedafed None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:50.786134 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20df8ed1-e510-4a87-b4eb-5bdfadedafed None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:50.786134 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20df8ed1-e510-4a87-b4eb-5bdfadedafed None admin] PATH_INFO: `/v3/users/c7e2a0e1de7a4e298b9c98129f405488` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:50.786377 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e553a60-1f1c-4047-a471-38863b1bb9ed None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:50.792542 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:50.793402 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20df8ed1-e510-4a87-b4eb-5bdfadedafed None admin] RBAC: Authorizing `identity:get_user(user_id=c7e2a0e1de7a4e298b9c98129f405488)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:50.795540 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20df8ed1-e510-4a87-b4eb-5bdfadedafed None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:50.797622 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 153/305] 10.222.0.16 () {56 vars in 1177 bytes} [Wed Aug 24 09:07:50 2022] GET /identity/v3/users/c7e2a0e1de7a4e298b9c98129f405488 => generated 293 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:07:50.798389 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20df8ed1-e510-4a87-b4eb-5bdfadedafed None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:50.889434 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e553a60-1f1c-4047-a471-38863b1bb9ed None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:50.909254 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20df8ed1-e510-4a87-b4eb-5bdfadedafed None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:50.924812 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacb3bac-a47a-4bf6-9fd8-e6f20b57d1b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:50.925417 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacb3bac-a47a-4bf6-9fd8-e6f20b57d1b5 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:50.925701 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacb3bac-a47a-4bf6-9fd8-e6f20b57d1b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:50.925975 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacb3bac-a47a-4bf6-9fd8-e6f20b57d1b5 None admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:50.926639 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacb3bac-a47a-4bf6-9fd8-e6f20b57d1b5 None admin] RBAC: Adding query filter params (name=member) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:07:50.926935 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacb3bac-a47a-4bf6-9fd8-e6f20b57d1b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:50.928342 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacb3bac-a47a-4bf6-9fd8-e6f20b57d1b5 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:50.936861 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 153/306] 10.222.0.16 () {56 vars in 1101 bytes} [Wed Aug 24 09:07:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:07:50.937714 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacb3bac-a47a-4bf6-9fd8-e6f20b57d1b5 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:50.948854 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa6cb34-2d1b-4f9f-92a7-6b816a1fcd36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:50.949478 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa6cb34-2d1b-4f9f-92a7-6b816a1fcd36 None admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:50.949761 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa6cb34-2d1b-4f9f-92a7-6b816a1fcd36 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:50.950034 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa6cb34-2d1b-4f9f-92a7-6b816a1fcd36 None admin] PATH_INFO: `/v3/groups` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:50.951863 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa6cb34-2d1b-4f9f-92a7-6b816a1fcd36 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:50.953621 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa6cb34-2d1b-4f9f-92a7-6b816a1fcd36 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:51.010172 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:51.020453 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 154/307] 10.222.0.16 () {60 vars in 1134 bytes} [Wed Aug 24 09:07:50 2022] POST /identity/v3/groups => generated 229 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:07:51.021276 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa6cb34-2d1b-4f9f-92a7-6b816a1fcd36 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:51.062200 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacb3bac-a47a-4bf6-9fd8-e6f20b57d1b5 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:51.076386 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:51.109509 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495154ea-ecad-48db-a8ee-c989eff9ce8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:51.110276 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495154ea-ecad-48db-a8ee-c989eff9ce8a None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:51.110653 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495154ea-ecad-48db-a8ee-c989eff9ce8a None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:51.110941 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495154ea-ecad-48db-a8ee-c989eff9ce8a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:51.111767 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495154ea-ecad-48db-a8ee-c989eff9ce8a None admin] RBAC: Adding query filter params (role.id=5a55760968cc43ef86318e5bec24f4ef, user.id=c7e2a0e1de7a4e298b9c98129f405488, scope.system=all) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:07:51.112082 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495154ea-ecad-48db-a8ee-c989eff9ce8a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:51.113676 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495154ea-ecad-48db-a8ee-c989eff9ce8a None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:51.125019 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 154/308] 10.222.0.16 () {56 vars in 1308 bytes} [Wed Aug 24 09:07:50 2022] GET /identity/v3/role_assignments?user.id=c7e2a0e1de7a4e298b9c98129f405488&scope.system=all&role.id=5a55760968cc43ef86318e5bec24f4ef => generated 226 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:07:51.165055 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa6cb34-2d1b-4f9f-92a7-6b816a1fcd36 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:51.209161 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4964c69d-3284-4e39-b20b-876ac34dd76f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:51.209875 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4964c69d-3284-4e39-b20b-876ac34dd76f None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:51.210163 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4964c69d-3284-4e39-b20b-876ac34dd76f None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:51.210442 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4964c69d-3284-4e39-b20b-876ac34dd76f None admin] PATH_INFO: `/v3/users/db38a9feea854ecfbb16c49e37d7c6d4` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:51.211671 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4964c69d-3284-4e39-b20b-876ac34dd76f None admin] RBAC: Authorizing `identity:get_user(user_id=db38a9feea854ecfbb16c49e37d7c6d4)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:51.213279 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4964c69d-3284-4e39-b20b-876ac34dd76f None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:51.215449 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 155/309] 10.222.0.16 () {56 vars in 1177 bytes} [Wed Aug 24 09:07:51 2022] GET /identity/v3/users/db38a9feea854ecfbb16c49e37d7c6d4 => generated 293 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:07:51.221599 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495154ea-ecad-48db-a8ee-c989eff9ce8a None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:51.344713 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495154ea-ecad-48db-a8ee-c989eff9ce8a None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:51.380083 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff490d8-764a-4239-be92-5547ec40ebac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:51.380736 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff490d8-764a-4239-be92-5547ec40ebac None admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:51.381014 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff490d8-764a-4239-be92-5547ec40ebac None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:51.381287 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff490d8-764a-4239-be92-5547ec40ebac None admin] PATH_INFO: `/v3/system/users/db38a9feea854ecfbb16c49e37d7c6d4/roles/136cdf78e00a426997dd4e37faff489b` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:51.382691 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff490d8-764a-4239-be92-5547ec40ebac None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=db38a9feea854ecfbb16c49e37d7c6d4, role_id=136cdf78e00a426997dd4e37faff489b)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:51.384100 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff490d8-764a-4239-be92-5547ec40ebac None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:51.385633 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:51.483939 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 155/310] 10.222.0.16 () {58 vars in 1334 bytes} [Wed Aug 24 09:07:51 2022] PUT /identity/v3/system/users/db38a9feea854ecfbb16c49e37d7c6d4/roles/136cdf78e00a426997dd4e37faff489b => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 09:07:51.561720 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf6c709-dee4-484d-8a3d-ebf7e926ec94 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:51.561894 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf6c709-dee4-484d-8a3d-ebf7e926ec94 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:51.562164 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf6c709-dee4-484d-8a3d-ebf7e926ec94 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:51.563628 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 156/311] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:07:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:07:51.569559 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be83711a-439f-41b2-833a-e232fcd3fec8 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:51.569853 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be83711a-439f-41b2-833a-e232fcd3fec8 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:51.570118 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be83711a-439f-41b2-833a-e232fcd3fec8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:51.590530 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-be83711a-439f-41b2-833a-e232fcd3fec8 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:07:51.591809 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-be83711a-439f-41b2-833a-e232fcd3fec8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:07:51.675011 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 156/312] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:07:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:07:51.949416 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf6c709-dee4-484d-8a3d-ebf7e926ec94 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:52.059819 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf6c709-dee4-484d-8a3d-ebf7e926ec94 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:52.070842 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:52.095120 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521eb1a6-7867-4b99-aba5-c6069ca016b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:52.097903 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521eb1a6-7867-4b99-aba5-c6069ca016b2 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:52.097903 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521eb1a6-7867-4b99-aba5-c6069ca016b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:52.097903 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521eb1a6-7867-4b99-aba5-c6069ca016b2 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:52.108370 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521eb1a6-7867-4b99-aba5-c6069ca016b2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:07:52.113304 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-521eb1a6-7867-4b99-aba5-c6069ca016b2 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 09:07:52.113304 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:07:52.113304 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:07:52.113304 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:07:52.113304 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:07:52.113304 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:07:52.113304 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:07:52.113304 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:07:52.113304 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:07:52.113304 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:07:52.113304 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:07:52.113304 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:07:52.113304 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:07:52.113304 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:07:52.113304 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:07:52.113304 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:07:52.114197 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:07:52.114197 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:07:52.114197 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:07:52.114197 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:07:52.114197 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:07:52.114197 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:07:52.114197 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:07:52.114197 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:07:52.114197 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:07:52.114197 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:07:52.114197 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:07:52.114197 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 24 09:07:52.114197 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:07:52.114197 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:07:52.114197 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:07:52.114197 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:07:52.115008 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:07:52.115008 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:07:52.115008 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:07:52.115008 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:07:52.115008 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:07:52.115008 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:07:52.115008 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:07:52.115008 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:07:52.115008 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:07:52.115008 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:07:52.115008 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 09:07:52.115008 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 24 09:07:52.115008 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 157/313] 10.222.0.16 () {56 vars in 1099 bytes} [Wed Aug 24 09:07:51 2022] GET /identity/v3/roles/reader => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:07:52.121611 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be83711a-439f-41b2-833a-e232fcd3fec8 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:52.231436 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be83711a-439f-41b2-833a-e232fcd3fec8 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:52.307056 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:52.315629 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a87aee6-2e3a-4505-8248-ce7d15d31dd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:52.316254 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a87aee6-2e3a-4505-8248-ce7d15d31dd6 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:52.316536 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a87aee6-2e3a-4505-8248-ce7d15d31dd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:52.322405 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a87aee6-2e3a-4505-8248-ce7d15d31dd6 None admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:52.322405 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a87aee6-2e3a-4505-8248-ce7d15d31dd6 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:07:52.322405 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a87aee6-2e3a-4505-8248-ce7d15d31dd6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:52.322405 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a87aee6-2e3a-4505-8248-ce7d15d31dd6 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:52.327907 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 157/314] 10.222.0.16 () {56 vars in 1101 bytes} [Wed Aug 24 09:07:52 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:07:52.333359 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521eb1a6-7867-4b99-aba5-c6069ca016b2 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:52.450468 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521eb1a6-7867-4b99-aba5-c6069ca016b2 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:52.485414 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa62936f-557a-4c6f-8c49-e57e1de3fa5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:52.485946 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa62936f-557a-4c6f-8c49-e57e1de3fa5f None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:52.486231 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa62936f-557a-4c6f-8c49-e57e1de3fa5f None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:52.486506 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa62936f-557a-4c6f-8c49-e57e1de3fa5f None admin] PATH_INFO: `/v3/users/6a5fc2cfb6c44b4cb49a33930d70d857` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:52.487684 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa62936f-557a-4c6f-8c49-e57e1de3fa5f None admin] RBAC: Authorizing `identity:get_user(user_id=6a5fc2cfb6c44b4cb49a33930d70d857)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:52.489141 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa62936f-557a-4c6f-8c49-e57e1de3fa5f None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:52.491040 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 158/315] 10.222.0.16 () {56 vars in 1177 bytes} [Wed Aug 24 09:07:52 2022] GET /identity/v3/users/6a5fc2cfb6c44b4cb49a33930d70d857 => generated 297 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:07:52.496559 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a87aee6-2e3a-4505-8248-ce7d15d31dd6 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:52.608153 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a87aee6-2e3a-4505-8248-ce7d15d31dd6 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:52.643994 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa024bec-af7f-4795-82cf-1309ca6c544c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:52.644630 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa024bec-af7f-4795-82cf-1309ca6c544c None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:52.644960 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa024bec-af7f-4795-82cf-1309ca6c544c None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:52.645240 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa024bec-af7f-4795-82cf-1309ca6c544c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:52.706966 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:52.717300 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa024bec-af7f-4795-82cf-1309ca6c544c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:52.718779 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa024bec-af7f-4795-82cf-1309ca6c544c None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:52.823230 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 158/316] 10.222.0.16 () {58 vars in 1282 bytes} [Wed Aug 24 09:07:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 09:07:52.829212 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa62936f-557a-4c6f-8c49-e57e1de3fa5f None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:52.871782 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:52.937069 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa62936f-557a-4c6f-8c49-e57e1de3fa5f None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:52.973566 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c17a67b-b6d3-4f64-b3dc-553afd9e844a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:52.975028 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c17a67b-b6d3-4f64-b3dc-553afd9e844a None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:52.975511 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c17a67b-b6d3-4f64-b3dc-553afd9e844a None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:52.976008 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c17a67b-b6d3-4f64-b3dc-553afd9e844a None admin] PATH_INFO: `/v3/projects/99f7f068dc714797ac5d88928535f7a2` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:52.977375 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c17a67b-b6d3-4f64-b3dc-553afd9e844a None admin] RBAC: Authorizing `identity:get_project(project_id=99f7f068dc714797ac5d88928535f7a2)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:52.979020 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c17a67b-b6d3-4f64-b3dc-553afd9e844a None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:52.981181 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 159/317] 10.222.0.16 () {56 vars in 1186 bytes} [Wed Aug 24 09:07:52 2022] GET /identity/v3/projects/99f7f068dc714797ac5d88928535f7a2 => generated 302 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:07:52.988686 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa024bec-af7f-4795-82cf-1309ca6c544c None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:53.086575 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:53.097581 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa024bec-af7f-4795-82cf-1309ca6c544c None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:53.135726 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d43371f-6212-467d-aaa8-68665398c522 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:53.136840 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da25a41e-1a1d-42eb-9546-ba2bcacae66f None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:53.137118 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da25a41e-1a1d-42eb-9546-ba2bcacae66f None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:53.137387 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da25a41e-1a1d-42eb-9546-ba2bcacae66f None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:53.138325 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d43371f-6212-467d-aaa8-68665398c522 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:53.138804 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d43371f-6212-467d-aaa8-68665398c522 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:53.139271 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d43371f-6212-467d-aaa8-68665398c522 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:53.140248 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d43371f-6212-467d-aaa8-68665398c522 None admin] RBAC: Adding query filter params (role.id=5a55760968cc43ef86318e5bec24f4ef, scope.project.id=99f7f068dc714797ac5d88928535f7a2, user.id=6a5fc2cfb6c44b4cb49a33930d70d857) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:07:53.140762 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d43371f-6212-467d-aaa8-68665398c522 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:53.141489 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 160/318] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:07:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:07:53.142670 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d43371f-6212-467d-aaa8-68665398c522 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:53.144348 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005609e8-ee02-4cc7-8fde-f7f0e98d7587 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:53.144693 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005609e8-ee02-4cc7-8fde-f7f0e98d7587 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:53.144940 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005609e8-ee02-4cc7-8fde-f7f0e98d7587 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:53.146158 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 161/319] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:07:53 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:07:53.147223 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7004ff49-d0ed-400d-adcd-c59888e607a7 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:53.147565 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7004ff49-d0ed-400d-adcd-c59888e607a7 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:53.147866 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7004ff49-d0ed-400d-adcd-c59888e607a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:53.150040 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 159/320] 10.222.0.16 () {56 vars in 1374 bytes} [Wed Aug 24 09:07:52 2022] GET /identity/v3/role_assignments?user.id=6a5fc2cfb6c44b4cb49a33930d70d857&scope.project.id=99f7f068dc714797ac5d88928535f7a2&role.id=5a55760968cc43ef86318e5bec24f4ef => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:07:53.155440 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c636f3d-7414-4145-882f-b6a1100b2430 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:53.155440 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c636f3d-7414-4145-882f-b6a1100b2430 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:53.155440 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c636f3d-7414-4145-882f-b6a1100b2430 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:53.169956 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-7004ff49-d0ed-400d-adcd-c59888e607a7 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:07:53.171184 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-7004ff49-d0ed-400d-adcd-c59888e607a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:07:53.173949 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-9c636f3d-7414-4145-882f-b6a1100b2430 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:07:53.175176 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-9c636f3d-7414-4145-882f-b6a1100b2430 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:07:53.250468 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 162/321] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:07:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:07:53.258334 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 160/322] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:07:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 09:07:53.516277 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a589bd5-c17c-4c43-9c65-7b92a6a21330 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:53.516627 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a589bd5-c17c-4c43-9c65-7b92a6a21330 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:53.516807 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a589bd5-c17c-4c43-9c65-7b92a6a21330 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:53.518147 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 163/323] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:07:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:07:53.523311 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9a4bec-2e58-4420-9b7a-5f762b5045ea None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:53.523621 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9a4bec-2e58-4420-9b7a-5f762b5045ea None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:53.523905 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9a4bec-2e58-4420-9b7a-5f762b5045ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:53.531511 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a589bd5-c17c-4c43-9c65-7b92a6a21330 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:53.542606 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-0c9a4bec-2e58-4420-9b7a-5f762b5045ea None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:07:53.543859 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-0c9a4bec-2e58-4420-9b7a-5f762b5045ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:07:53.569307 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:53.610992 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 161/324] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:07:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:07:53.612146 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9a4bec-2e58-4420-9b7a-5f762b5045ea None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:53.627416 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:53.641497 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a589bd5-c17c-4c43-9c65-7b92a6a21330 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:53.678383 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5983dbaf-eee1-4279-abe4-6acfc737a184 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:53.679008 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5983dbaf-eee1-4279-abe4-6acfc737a184 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:53.679286 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5983dbaf-eee1-4279-abe4-6acfc737a184 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:53.679551 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5983dbaf-eee1-4279-abe4-6acfc737a184 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:53.680679 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5983dbaf-eee1-4279-abe4-6acfc737a184 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:53.682134 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5983dbaf-eee1-4279-abe4-6acfc737a184 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:53.683862 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 164/325] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 24 09:07:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:07:53.689654 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5983dbaf-eee1-4279-abe4-6acfc737a184 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:53.718557 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9a4bec-2e58-4420-9b7a-5f762b5045ea None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:53.755508 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9bae80-8af1-4083-803b-0315c7e7d428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:53.756176 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9bae80-8af1-4083-803b-0315c7e7d428 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:53.756486 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9bae80-8af1-4083-803b-0315c7e7d428 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:53.756758 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9bae80-8af1-4083-803b-0315c7e7d428 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:53.768670 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9bae80-8af1-4083-803b-0315c7e7d428 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:07:53.773458 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-ab9bae80-8af1-4083-803b-0315c7e7d428 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 09:07:53.773458 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:07:53.773458 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:07:53.773458 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:07:53.773458 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:07:53.773458 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:07:53.773458 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:07:53.773458 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:07:53.773458 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:07:53.773458 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:07:53.773458 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:07:53.773458 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:07:53.773458 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:07:53.773458 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:07:53.773458 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:07:53.773458 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:07:53.774343 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:07:53.774343 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:07:53.774343 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:07:53.774343 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:07:53.774343 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:07:53.774343 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:07:53.774343 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:07:53.774343 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:07:53.774343 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:07:53.774343 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:07:53.774343 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:07:53.774343 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Aug 24 09:07:53.774343 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:07:53.774343 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:07:53.774343 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:07:53.774343 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:07:53.775360 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:07:53.775360 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:07:53.775360 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:07:53.775360 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:07:53.775360 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:07:53.775360 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:07:53.775360 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:07:53.775360 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:07:53.775360 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:07:53.775360 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:07:53.775360 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 09:07:53.775360 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Aug 24 09:07:53.775360 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 162/326] 10.222.0.16 () {56 vars in 1099 bytes} [Wed Aug 24 09:07:53 2022] GET /identity/v3/roles/reader => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:07:53.782912 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9bae80-8af1-4083-803b-0315c7e7d428 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:53.799438 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5983dbaf-eee1-4279-abe4-6acfc737a184 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:53.802589 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:53.836296 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a90c8bc-741c-4313-959e-114cd49294f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:53.836899 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a90c8bc-741c-4313-959e-114cd49294f5 None admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:53.837176 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a90c8bc-741c-4313-959e-114cd49294f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:53.837446 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a90c8bc-741c-4313-959e-114cd49294f5 None admin] PATH_INFO: `/v3/groups` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:53.838067 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a90c8bc-741c-4313-959e-114cd49294f5 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:53.839545 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a90c8bc-741c-4313-959e-114cd49294f5 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:53.860592 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:53.885444 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9bae80-8af1-4083-803b-0315c7e7d428 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:53.918287 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c50aa9-e988-4094-aa02-25abebc15ecf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:53.919407 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c50aa9-e988-4094-aa02-25abebc15ecf None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:53.919721 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c50aa9-e988-4094-aa02-25abebc15ecf None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:53.919995 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c50aa9-e988-4094-aa02-25abebc15ecf None admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:53.920652 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c50aa9-e988-4094-aa02-25abebc15ecf None admin] RBAC: Adding query filter params (name=reader) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:07:53.920958 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c50aa9-e988-4094-aa02-25abebc15ecf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:53.922343 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c50aa9-e988-4094-aa02-25abebc15ecf None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:53.930074 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 163/327] 10.222.0.16 () {56 vars in 1101 bytes} [Wed Aug 24 09:07:53 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:07:53.930850 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c50aa9-e988-4094-aa02-25abebc15ecf None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:53.945112 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 165/328] 10.222.0.16 () {60 vars in 1134 bytes} [Wed Aug 24 09:07:53 2022] POST /identity/v3/groups => generated 226 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 09:07:53.945112 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a90c8bc-741c-4313-959e-114cd49294f5 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:54.050944 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c50aa9-e988-4094-aa02-25abebc15ecf None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:54.055159 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a90c8bc-741c-4313-959e-114cd49294f5 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:54.090645 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b990c3e-8982-4575-82f9-28a08fad9076 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:54.091669 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b990c3e-8982-4575-82f9-28a08fad9076 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:54.091669 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b990c3e-8982-4575-82f9-28a08fad9076 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:54.091873 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b990c3e-8982-4575-82f9-28a08fad9076 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:54.095532 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24d4d83-1a95-4f31-bc8a-51a47b8c0de9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:54.096223 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24d4d83-1a95-4f31-bc8a-51a47b8c0de9 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:54.096508 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24d4d83-1a95-4f31-bc8a-51a47b8c0de9 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:54.096783 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24d4d83-1a95-4f31-bc8a-51a47b8c0de9 None admin] PATH_INFO: `/v3/users/c7e2a0e1de7a4e298b9c98129f405488` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:54.098650 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:54.098792 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24d4d83-1a95-4f31-bc8a-51a47b8c0de9 None admin] RBAC: Authorizing `identity:get_user(user_id=c7e2a0e1de7a4e298b9c98129f405488)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:54.101651 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24d4d83-1a95-4f31-bc8a-51a47b8c0de9 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:54.101916 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 166/329] 10.222.0.16 () {56 vars in 1177 bytes} [Wed Aug 24 09:07:53 2022] GET /identity/v3/users/c7e2a0e1de7a4e298b9c98129f405488 => generated 293 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:07:54.107775 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b990c3e-8982-4575-82f9-28a08fad9076 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:07:54.108093 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24d4d83-1a95-4f31-bc8a-51a47b8c0de9 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:54.112719 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-1b990c3e-8982-4575-82f9-28a08fad9076 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 09:07:54.112719 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:07:54.112719 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:07:54.112719 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:07:54.112719 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:07:54.112719 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:07:54.112719 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:07:54.112719 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:07:54.112719 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:07:54.112719 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:07:54.112719 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:07:54.112719 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:07:54.112719 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:07:54.112719 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:07:54.112719 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:07:54.112719 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:07:54.113642 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:07:54.113642 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:07:54.113642 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:07:54.113642 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:07:54.113642 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:07:54.113642 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:07:54.113642 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:07:54.113642 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:07:54.113642 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:07:54.113642 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:07:54.113642 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:07:54.113642 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Aug 24 09:07:54.113642 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:07:54.113642 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:07:54.113642 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:07:54.113642 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:07:54.114440 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:07:54.114440 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:07:54.114440 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:07:54.114440 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:07:54.114440 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:07:54.114440 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:07:54.114440 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:07:54.114440 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:07:54.114440 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:07:54.114440 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:07:54.114440 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 09:07:54.114440 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Aug 24 09:07:54.114440 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 164/330] 10.222.0.16 () {56 vars in 1099 bytes} [Wed Aug 24 09:07:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:07:54.122549 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b990c3e-8982-4575-82f9-28a08fad9076 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:54.148157 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:54.233742 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24d4d83-1a95-4f31-bc8a-51a47b8c0de9 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:54.249029 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b990c3e-8982-4575-82f9-28a08fad9076 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:54.277658 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d829cf-fb53-4526-913d-683ca63a15cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:54.277658 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d829cf-fb53-4526-913d-683ca63a15cc None admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:54.277658 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d829cf-fb53-4526-913d-683ca63a15cc None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:54.277658 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d829cf-fb53-4526-913d-683ca63a15cc None admin] PATH_INFO: `/v3/system/users/c7e2a0e1de7a4e298b9c98129f405488/roles/5a55760968cc43ef86318e5bec24f4ef` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:54.284109 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d829cf-fb53-4526-913d-683ca63a15cc None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=c7e2a0e1de7a4e298b9c98129f405488, role_id=5a55760968cc43ef86318e5bec24f4ef)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:54.284109 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d829cf-fb53-4526-913d-683ca63a15cc None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:54.292190 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae38076-f3f4-4e6b-9c1c-306947fc651d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:54.293121 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae38076-f3f4-4e6b-9c1c-306947fc651d None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:54.293622 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae38076-f3f4-4e6b-9c1c-306947fc651d None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:54.294468 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae38076-f3f4-4e6b-9c1c-306947fc651d None admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:54.297098 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae38076-f3f4-4e6b-9c1c-306947fc651d None admin] RBAC: Adding query filter params (name=member) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:07:54.297098 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae38076-f3f4-4e6b-9c1c-306947fc651d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:54.297098 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae38076-f3f4-4e6b-9c1c-306947fc651d None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:54.309451 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 165/331] 10.222.0.16 () {56 vars in 1101 bytes} [Wed Aug 24 09:07:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:07:54.314631 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae38076-f3f4-4e6b-9c1c-306947fc651d None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:54.344909 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 167/332] 10.222.0.16 () {58 vars in 1334 bytes} [Wed Aug 24 09:07:54 2022] PUT /identity/v3/system/users/c7e2a0e1de7a4e298b9c98129f405488/roles/5a55760968cc43ef86318e5bec24f4ef => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 09:07:54.392990 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:54.440222 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae38076-f3f4-4e6b-9c1c-306947fc651d None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:54.483789 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97f8dfc-0adf-4dd3-8064-ab1856583b09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:54.484688 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97f8dfc-0adf-4dd3-8064-ab1856583b09 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:54.485109 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97f8dfc-0adf-4dd3-8064-ab1856583b09 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:54.485838 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97f8dfc-0adf-4dd3-8064-ab1856583b09 None admin] PATH_INFO: `/v3/users/db38a9feea854ecfbb16c49e37d7c6d4` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:54.487208 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97f8dfc-0adf-4dd3-8064-ab1856583b09 None admin] RBAC: Authorizing `identity:get_user(user_id=db38a9feea854ecfbb16c49e37d7c6d4)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:54.488905 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97f8dfc-0adf-4dd3-8064-ab1856583b09 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:54.491042 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 166/333] 10.222.0.16 () {56 vars in 1177 bytes} [Wed Aug 24 09:07:54 2022] GET /identity/v3/users/db38a9feea854ecfbb16c49e37d7c6d4 => generated 293 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:07:54.497050 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d829cf-fb53-4526-913d-683ca63a15cc None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:54.600879 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:54.607489 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d829cf-fb53-4526-913d-683ca63a15cc None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:54.644981 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1717ca21-60a1-4b84-80b0-5873e71fa6c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:54.645823 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1717ca21-60a1-4b84-80b0-5873e71fa6c0 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:54.646242 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1717ca21-60a1-4b84-80b0-5873e71fa6c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:54.646679 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1717ca21-60a1-4b84-80b0-5873e71fa6c0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:54.647681 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1717ca21-60a1-4b84-80b0-5873e71fa6c0 None admin] RBAC: Adding query filter params (role.id=136cdf78e00a426997dd4e37faff489b, user.id=db38a9feea854ecfbb16c49e37d7c6d4, scope.system=all) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:07:54.648127 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1717ca21-60a1-4b84-80b0-5873e71fa6c0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:54.649740 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1717ca21-60a1-4b84-80b0-5873e71fa6c0 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:54.667372 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 168/334] 10.222.0.16 () {56 vars in 1308 bytes} [Wed Aug 24 09:07:54 2022] GET /identity/v3/role_assignments?user.id=db38a9feea854ecfbb16c49e37d7c6d4&scope.system=all&role.id=136cdf78e00a426997dd4e37faff489b => generated 510 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:07:56.088047 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7980399-48f5-43b4-ba5e-649535bf9206 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:56.089078 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7980399-48f5-43b4-ba5e-649535bf9206 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:56.089557 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7980399-48f5-43b4-ba5e-649535bf9206 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:56.091535 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 167/335] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:07:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:07:56.097991 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e94f6ba-dac2-4926-80fd-3513e71f5a05 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:56.099801 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e94f6ba-dac2-4926-80fd-3513e71f5a05 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:56.100287 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e94f6ba-dac2-4926-80fd-3513e71f5a05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:56.125528 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-4e94f6ba-dac2-4926-80fd-3513e71f5a05 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:07:56.127034 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-4e94f6ba-dac2-4926-80fd-3513e71f5a05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:07:56.203322 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 169/336] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:07:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:07:56.582027 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1db08d-5a1e-4cf2-ba4b-6dd965bc5d30 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:56.582408 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1db08d-5a1e-4cf2-ba4b-6dd965bc5d30 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:56.582586 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1db08d-5a1e-4cf2-ba4b-6dd965bc5d30 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:56.584345 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 168/337] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:07:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:07:56.589813 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d6ddd0-7267-4d9a-935c-bbd977781f11 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:56.590102 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d6ddd0-7267-4d9a-935c-bbd977781f11 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:56.590379 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d6ddd0-7267-4d9a-935c-bbd977781f11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:56.616027 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-b4d6ddd0-7267-4d9a-935c-bbd977781f11 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:07:56.617220 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-b4d6ddd0-7267-4d9a-935c-bbd977781f11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:07:56.705594 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 170/338] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:07:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 09:07:56.767017 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1db08d-5a1e-4cf2-ba4b-6dd965bc5d30 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:56.806348 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8935f4-9151-41cf-ba84-4191ba2544f2 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:56.806671 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8935f4-9151-41cf-ba84-4191ba2544f2 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:56.806894 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8935f4-9151-41cf-ba84-4191ba2544f2 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:56.808766 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 171/339] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:07:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:07:56.814613 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6216a59-79c3-4afb-8906-4ade82e78549 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:56.815003 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6216a59-79c3-4afb-8906-4ade82e78549 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:56.815381 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6216a59-79c3-4afb-8906-4ade82e78549 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:56.846603 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-a6216a59-79c3-4afb-8906-4ade82e78549 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:07:56.848064 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-a6216a59-79c3-4afb-8906-4ade82e78549 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:07:56.872610 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:56.952082 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1db08d-5a1e-4cf2-ba4b-6dd965bc5d30 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:56.984739 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:56.993479 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703fb1af-ef68-47a9-9f66-58ae2a3058e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:56.994214 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703fb1af-ef68-47a9-9f66-58ae2a3058e1 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:56.994548 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703fb1af-ef68-47a9-9f66-58ae2a3058e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:56.994816 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703fb1af-ef68-47a9-9f66-58ae2a3058e1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:56.997600 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 172/340] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:07:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:07:56.999043 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a3654a-89a6-4478-bae5-430ae935a7e3 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:56.999483 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a3654a-89a6-4478-bae5-430ae935a7e3 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:56.999931 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a3654a-89a6-4478-bae5-430ae935a7e3 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:57.003365 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 173/341] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:07:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:07:57.008995 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96230a4e-91f1-4131-8b27-30a183a7ea44 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:57.009530 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96230a4e-91f1-4131-8b27-30a183a7ea44 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:57.010016 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96230a4e-91f1-4131-8b27-30a183a7ea44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:57.010986 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703fb1af-ef68-47a9-9f66-58ae2a3058e1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:07:57.016265 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-703fb1af-ef68-47a9-9f66-58ae2a3058e1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 09:07:57.016265 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:07:57.016265 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:07:57.016265 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:07:57.016265 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:07:57.016265 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:07:57.016265 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:07:57.016265 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:07:57.016265 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:07:57.016265 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:07:57.016265 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:07:57.016265 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:07:57.016265 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:07:57.016265 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:07:57.016265 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:07:57.016265 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:07:57.017119 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:07:57.017119 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:07:57.017119 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:07:57.017119 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:07:57.017119 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:07:57.017119 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:07:57.017119 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:07:57.017119 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:07:57.017119 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:07:57.017119 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:07:57.017119 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:07:57.017119 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Aug 24 09:07:57.017119 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:07:57.017119 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:07:57.017119 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:07:57.017119 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:07:57.018457 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:07:57.018457 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:07:57.018457 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:07:57.018457 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:07:57.018457 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:07:57.018457 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:07:57.018457 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:07:57.018457 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:07:57.018457 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:07:57.018457 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:07:57.018457 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 09:07:57.018457 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Aug 24 09:07:57.018457 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 169/342] 10.222.0.16 () {56 vars in 1099 bytes} [Wed Aug 24 09:07:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:07:57.024734 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703fb1af-ef68-47a9-9f66-58ae2a3058e1 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:57.045365 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-96230a4e-91f1-4131-8b27-30a183a7ea44 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:07:57.046908 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-96230a4e-91f1-4131-8b27-30a183a7ea44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:07:57.163227 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703fb1af-ef68-47a9-9f66-58ae2a3058e1 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:57.164213 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 174/343] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:07:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 09:07:57.221375 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c21821-4647-49a6-ab27-2e320b84d8a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:57.222001 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c21821-4647-49a6-ab27-2e320b84d8a2 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:57.222284 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c21821-4647-49a6-ab27-2e320b84d8a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:57.222563 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c21821-4647-49a6-ab27-2e320b84d8a2 None admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:57.223242 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c21821-4647-49a6-ab27-2e320b84d8a2 None admin] RBAC: Adding query filter params (name=member) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:07:57.223555 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c21821-4647-49a6-ab27-2e320b84d8a2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:57.225142 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c21821-4647-49a6-ab27-2e320b84d8a2 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:57.239056 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 170/344] 10.222.0.16 () {56 vars in 1101 bytes} [Wed Aug 24 09:07:57 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:07:57.244682 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96230a4e-91f1-4131-8b27-30a183a7ea44 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:57.275903 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c21821-4647-49a6-ab27-2e320b84d8a2 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:57.359933 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:57.367823 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:57.394889 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96230a4e-91f1-4131-8b27-30a183a7ea44 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:57.433237 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c21821-4647-49a6-ab27-2e320b84d8a2 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:57.444889 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c0aab1-323f-47b2-970b-7b55e822b2fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:57.445522 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c0aab1-323f-47b2-970b-7b55e822b2fc None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:57.445806 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c0aab1-323f-47b2-970b-7b55e822b2fc None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:57.446084 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c0aab1-323f-47b2-970b-7b55e822b2fc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:57.489490 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55774cdc-6793-4783-956f-8aa281496ff0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:57.490148 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55774cdc-6793-4783-956f-8aa281496ff0 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:57.490432 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55774cdc-6793-4783-956f-8aa281496ff0 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:57.490708 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55774cdc-6793-4783-956f-8aa281496ff0 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:57.512910 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55774cdc-6793-4783-956f-8aa281496ff0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:07:57.518205 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-55774cdc-6793-4783-956f-8aa281496ff0 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 09:07:57.518205 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:07:57.518205 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:07:57.518205 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:07:57.518205 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:07:57.518205 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:07:57.518205 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:07:57.518205 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:07:57.518205 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:07:57.518205 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:07:57.518205 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:07:57.518205 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:07:57.518205 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:07:57.518205 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:07:57.518205 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:07:57.518205 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:07:57.519098 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:07:57.519098 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:07:57.519098 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:07:57.519098 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:07:57.519098 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:07:57.519098 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:07:57.519098 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:07:57.519098 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:07:57.519098 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:07:57.519098 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:07:57.519098 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:07:57.519098 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Aug 24 09:07:57.519098 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:07:57.519098 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:07:57.519098 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:07:57.519098 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:07:57.519905 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:07:57.519905 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:07:57.519905 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:07:57.519905 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:07:57.519905 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:07:57.519905 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:07:57.519905 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:07:57.519905 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:07:57.519905 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:07:57.519905 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:07:57.519905 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 09:07:57.519905 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Aug 24 09:07:57.521687 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 171/345] 10.222.0.16 () {56 vars in 1099 bytes} [Wed Aug 24 09:07:57 2022] GET /identity/v3/roles/reader => generated 84 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:07:57.527822 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55774cdc-6793-4783-956f-8aa281496ff0 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:57.565135 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c0aab1-323f-47b2-970b-7b55e822b2fc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:57.566611 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c0aab1-323f-47b2-970b-7b55e822b2fc None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:57.669855 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:57.670842 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55774cdc-6793-4783-956f-8aa281496ff0 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:57.692012 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:57.721358 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 175/346] 10.222.0.16 () {58 vars in 1282 bytes} [Wed Aug 24 09:07:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 477 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 09:07:57.723057 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c0aab1-323f-47b2-970b-7b55e822b2fc None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:57.743185 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d95df98-9b00-43c7-af27-ff4f295db28c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:57.744228 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d95df98-9b00-43c7-af27-ff4f295db28c None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:57.744513 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d95df98-9b00-43c7-af27-ff4f295db28c None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:57.744790 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d95df98-9b00-43c7-af27-ff4f295db28c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:57.765755 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d95df98-9b00-43c7-af27-ff4f295db28c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:07:57.771994 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-8d95df98-9b00-43c7-af27-ff4f295db28c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 09:07:57.771994 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:07:57.771994 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:07:57.771994 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:07:57.771994 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:07:57.771994 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:07:57.771994 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:07:57.771994 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:07:57.771994 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:07:57.771994 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:07:57.771994 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:07:57.771994 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:07:57.771994 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:07:57.771994 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:07:57.771994 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:07:57.771994 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:07:57.772963 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:07:57.772963 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:07:57.772963 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:07:57.772963 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:07:57.772963 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:07:57.772963 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:07:57.772963 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:07:57.772963 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:07:57.772963 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:07:57.772963 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:07:57.772963 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:07:57.772963 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Aug 24 09:07:57.772963 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:07:57.772963 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:07:57.772963 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:07:57.772963 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:07:57.773779 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:07:57.773779 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:07:57.773779 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:07:57.773779 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:07:57.773779 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:07:57.773779 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:07:57.773779 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:07:57.773779 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:07:57.773779 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:07:57.773779 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:07:57.773779 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 09:07:57.773779 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Aug 24 09:07:57.779777 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 172/347] 10.222.0.16 () {56 vars in 1114 bytes} [Wed Aug 24 09:07:57 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:07:57.780860 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d95df98-9b00-43c7-af27-ff4f295db28c None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:57.861590 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c0aab1-323f-47b2-970b-7b55e822b2fc None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:57.906589 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909d5262-a7f1-4617-b423-34b28770bef0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:57.907824 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909d5262-a7f1-4617-b423-34b28770bef0 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:57.908302 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909d5262-a7f1-4617-b423-34b28770bef0 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:57.908766 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909d5262-a7f1-4617-b423-34b28770bef0 None admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:57.909622 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909d5262-a7f1-4617-b423-34b28770bef0 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:07:57.910117 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909d5262-a7f1-4617-b423-34b28770bef0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:57.910668 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d95df98-9b00-43c7-af27-ff4f295db28c None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:57.912579 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909d5262-a7f1-4617-b423-34b28770bef0 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:57.922901 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 176/348] 10.222.0.16 () {56 vars in 1101 bytes} [Wed Aug 24 09:07:57 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:07:57.924174 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909d5262-a7f1-4617-b423-34b28770bef0 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:57.952042 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ecd71d-48ab-4160-a351-d72edaf37cea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:57.952988 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ecd71d-48ab-4160-a351-d72edaf37cea None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:57.952988 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ecd71d-48ab-4160-a351-d72edaf37cea None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:57.953152 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ecd71d-48ab-4160-a351-d72edaf37cea None admin] PATH_INFO: `/v3/roles/member` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:57.966385 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ecd71d-48ab-4160-a351-d72edaf37cea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:07:57.966840 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:57.972461 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-13ecd71d-48ab-4160-a351-d72edaf37cea None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 09:07:57.972461 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:07:57.972461 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:07:57.972461 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:07:57.972461 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:07:57.972461 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:07:57.972461 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:07:57.972461 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:07:57.972461 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:07:57.972461 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:07:57.972461 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:07:57.972461 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:07:57.972461 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:07:57.972461 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:07:57.972461 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:07:57.972461 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:07:57.973360 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:07:57.973360 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:07:57.973360 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:07:57.973360 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:07:57.973360 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:07:57.973360 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:07:57.973360 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:07:57.973360 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:07:57.973360 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:07:57.973360 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:07:57.973360 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:07:57.973360 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Aug 24 09:07:57.973360 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:07:57.973360 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:07:57.973360 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:07:57.973360 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:07:57.974288 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:07:57.974288 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:07:57.974288 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:07:57.974288 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:07:57.974288 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:07:57.974288 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:07:57.974288 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:07:57.974288 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:07:57.974288 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:07:57.974288 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:07:57.974288 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 09:07:57.974288 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Aug 24 09:07:57.982015 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 173/349] 10.222.0.16 () {56 vars in 1099 bytes} [Wed Aug 24 09:07:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:07:57.982015 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ecd71d-48ab-4160-a351-d72edaf37cea None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:58.011068 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:58.053871 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909d5262-a7f1-4617-b423-34b28770bef0 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:58.098777 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ecd71d-48ab-4160-a351-d72edaf37cea None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:58.100482 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76d8cd8-5d46-4364-bb57-88127c86eab7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:58.101126 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76d8cd8-5d46-4364-bb57-88127c86eab7 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:58.101452 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76d8cd8-5d46-4364-bb57-88127c86eab7 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:58.101726 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76d8cd8-5d46-4364-bb57-88127c86eab7 None admin] PATH_INFO: `/v3/projects/7d6be68cf8b046d5b03fba3c5a367c11` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:58.102862 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76d8cd8-5d46-4364-bb57-88127c86eab7 None admin] RBAC: Authorizing `identity:get_project(project_id=7d6be68cf8b046d5b03fba3c5a367c11)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:58.104376 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76d8cd8-5d46-4364-bb57-88127c86eab7 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:58.106103 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 177/350] 10.222.0.16 () {56 vars in 1186 bytes} [Wed Aug 24 09:07:57 2022] GET /identity/v3/projects/7d6be68cf8b046d5b03fba3c5a367c11 => generated 298 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:07:58.107396 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76d8cd8-5d46-4364-bb57-88127c86eab7 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:58.137348 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3071b3a-d989-485f-a5fe-4699fd7611a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:58.137967 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3071b3a-d989-485f-a5fe-4699fd7611a1 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:58.138245 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3071b3a-d989-485f-a5fe-4699fd7611a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:58.138518 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3071b3a-d989-485f-a5fe-4699fd7611a1 None admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:58.139180 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3071b3a-d989-485f-a5fe-4699fd7611a1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:07:58.139478 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3071b3a-d989-485f-a5fe-4699fd7611a1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:58.140925 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3071b3a-d989-485f-a5fe-4699fd7611a1 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:58.154314 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 174/351] 10.222.0.16 () {56 vars in 1111 bytes} [Wed Aug 24 09:07:57 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:07:58.154999 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3071b3a-d989-485f-a5fe-4699fd7611a1 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:58.232898 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76d8cd8-5d46-4364-bb57-88127c86eab7 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:58.235210 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:58.264652 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:58.274955 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c417409d-386b-4d06-b06b-6a1835f7c2f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:58.275397 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3071b3a-d989-485f-a5fe-4699fd7611a1 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:58.275930 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c417409d-386b-4d06-b06b-6a1835f7c2f0 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:58.276215 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c417409d-386b-4d06-b06b-6a1835f7c2f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:58.276491 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c417409d-386b-4d06-b06b-6a1835f7c2f0 None admin] PATH_INFO: `/v3/users/c7e2a0e1de7a4e298b9c98129f405488` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:58.277694 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c417409d-386b-4d06-b06b-6a1835f7c2f0 None admin] RBAC: Authorizing `identity:get_user(user_id=c7e2a0e1de7a4e298b9c98129f405488)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:58.281772 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c417409d-386b-4d06-b06b-6a1835f7c2f0 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:58.281772 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 178/352] 10.222.0.16 () {56 vars in 1177 bytes} [Wed Aug 24 09:07:58 2022] GET /identity/v3/users/c7e2a0e1de7a4e298b9c98129f405488 => generated 293 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:07:58.284703 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508ec6eb-1d43-4049-b6d9-0e3cfcb519d6 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:58.284703 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508ec6eb-1d43-4049-b6d9-0e3cfcb519d6 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:58.284703 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508ec6eb-1d43-4049-b6d9-0e3cfcb519d6 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:58.284703 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 179/353] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:07:58 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:07:58.285815 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508ec6eb-1d43-4049-b6d9-0e3cfcb519d6 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:58.332515 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea49b94b-a9a5-49c6-8f57-00b45f58567e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:58.333133 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea49b94b-a9a5-49c6-8f57-00b45f58567e None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:58.333415 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea49b94b-a9a5-49c6-8f57-00b45f58567e None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:58.333686 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea49b94b-a9a5-49c6-8f57-00b45f58567e None admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:58.334359 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea49b94b-a9a5-49c6-8f57-00b45f58567e None admin] RBAC: Adding query filter params (name=member) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:07:58.334664 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea49b94b-a9a5-49c6-8f57-00b45f58567e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:58.336074 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea49b94b-a9a5-49c6-8f57-00b45f58567e None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:58.347625 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 175/354] 10.222.0.16 () {56 vars in 1101 bytes} [Wed Aug 24 09:07:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:07:58.348413 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea49b94b-a9a5-49c6-8f57-00b45f58567e None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:58.429703 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508ec6eb-1d43-4049-b6d9-0e3cfcb519d6 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:58.485322 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b294a71-a206-4575-b45c-8826324a1f15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:58.485887 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b294a71-a206-4575-b45c-8826324a1f15 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:58.486257 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b294a71-a206-4575-b45c-8826324a1f15 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:58.486523 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b294a71-a206-4575-b45c-8826324a1f15 None admin] PATH_INFO: `/v3/groups/28f5845ddc6847b38fd62490f9dce3d9` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:58.488825 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea49b94b-a9a5-49c6-8f57-00b45f58567e None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:58.494750 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b294a71-a206-4575-b45c-8826324a1f15 None admin] RBAC: Authorizing `identity:get_group(group_id=28f5845ddc6847b38fd62490f9dce3d9)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:58.496275 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b294a71-a206-4575-b45c-8826324a1f15 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:58.498712 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 180/355] 10.222.0.16 () {56 vars in 1180 bytes} [Wed Aug 24 09:07:58 2022] GET /identity/v3/groups/28f5845ddc6847b38fd62490f9dce3d9 => generated 226 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:07:58.500460 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2039c9-04a4-4366-b1eb-1b715dcca387 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:58.500754 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2039c9-04a4-4366-b1eb-1b715dcca387 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:58.501029 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2039c9-04a4-4366-b1eb-1b715dcca387 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:58.522586 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-4a2039c9-04a4-4366-b1eb-1b715dcca387 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:07:58.524510 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-4a2039c9-04a4-4366-b1eb-1b715dcca387 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:07:58.539017 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506daec8-d9e8-4f9c-978f-769fcddd99c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:58.540446 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506daec8-d9e8-4f9c-978f-769fcddd99c1 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:58.540446 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506daec8-d9e8-4f9c-978f-769fcddd99c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:58.540446 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506daec8-d9e8-4f9c-978f-769fcddd99c1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:58.566997 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:58.571705 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:58.622799 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 181/356] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:07:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:07:58.624170 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2039c9-04a4-4366-b1eb-1b715dcca387 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:58.632810 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506daec8-d9e8-4f9c-978f-769fcddd99c1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:58.634297 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506daec8-d9e8-4f9c-978f-769fcddd99c1 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:58.759157 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 176/357] 10.222.0.16 () {58 vars in 1282 bytes} [Wed Aug 24 09:07:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 09:07:58.760463 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506daec8-d9e8-4f9c-978f-769fcddd99c1 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:58.765696 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2039c9-04a4-4366-b1eb-1b715dcca387 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:58.812846 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c01bab4-1920-4799-ae1f-a046a181c193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:58.813810 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c01bab4-1920-4799-ae1f-a046a181c193 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:58.814451 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c01bab4-1920-4799-ae1f-a046a181c193 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:58.814920 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c01bab4-1920-4799-ae1f-a046a181c193 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:58.815908 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c01bab4-1920-4799-ae1f-a046a181c193 None admin] RBAC: Adding query filter params (role.id=5a55760968cc43ef86318e5bec24f4ef, user.id=c7e2a0e1de7a4e298b9c98129f405488, scope.system=all) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:07:58.816218 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c01bab4-1920-4799-ae1f-a046a181c193 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:58.817646 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c01bab4-1920-4799-ae1f-a046a181c193 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:58.827855 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:58.831814 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 182/358] 10.222.0.16 () {56 vars in 1308 bytes} [Wed Aug 24 09:07:58 2022] GET /identity/v3/role_assignments?user.id=c7e2a0e1de7a4e298b9c98129f405488&scope.system=all&role.id=5a55760968cc43ef86318e5bec24f4ef => generated 510 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:07:58.832787 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c01bab4-1920-4799-ae1f-a046a181c193 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:58.867939 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:58.897632 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506daec8-d9e8-4f9c-978f-769fcddd99c1 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:58.954282 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e055ba-a959-4e3d-913b-493e6e5b469d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:58.955188 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e055ba-a959-4e3d-913b-493e6e5b469d None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:58.955471 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e055ba-a959-4e3d-913b-493e6e5b469d None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:58.955778 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e055ba-a959-4e3d-913b-493e6e5b469d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:58.974171 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c01bab4-1920-4799-ae1f-a046a181c193 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:59.018179 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89ba56f-2785-48c0-a7ef-f8c6c3f1d134 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:59.019142 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89ba56f-2785-48c0-a7ef-f8c6c3f1d134 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:59.019425 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89ba56f-2785-48c0-a7ef-f8c6c3f1d134 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:59.019732 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89ba56f-2785-48c0-a7ef-f8c6c3f1d134 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:59.020471 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89ba56f-2785-48c0-a7ef-f8c6c3f1d134 None admin] RBAC: Adding query filter params (group.id=28f5845ddc6847b38fd62490f9dce3d9, role.id=136cdf78e00a426997dd4e37faff489b, scope.project.id=7d6be68cf8b046d5b03fba3c5a367c11) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:07:59.020780 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89ba56f-2785-48c0-a7ef-f8c6c3f1d134 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:59.022229 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89ba56f-2785-48c0-a7ef-f8c6c3f1d134 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:59.030428 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 183/359] 10.222.0.16 () {56 vars in 1376 bytes} [Wed Aug 24 09:07:58 2022] GET /identity/v3/role_assignments?group.id=28f5845ddc6847b38fd62490f9dce3d9&scope.project.id=7d6be68cf8b046d5b03fba3c5a367c11&role.id=136cdf78e00a426997dd4e37faff489b => generated 260 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:07:59.030962 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89ba56f-2785-48c0-a7ef-f8c6c3f1d134 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:59.049439 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e055ba-a959-4e3d-913b-493e6e5b469d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:59.051889 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e055ba-a959-4e3d-913b-493e6e5b469d None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:59.127673 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:59.160729 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:59.174399 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89ba56f-2785-48c0-a7ef-f8c6c3f1d134 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:59.192829 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 177/360] 10.222.0.16 () {58 vars in 1282 bytes} [Wed Aug 24 09:07:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 09:07:59.194640 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f447ac59-f782-40c6-ac78-cc48a5a2bb80 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:59.195073 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f447ac59-f782-40c6-ac78-cc48a5a2bb80 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:59.195528 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f447ac59-f782-40c6-ac78-cc48a5a2bb80 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:59.197099 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 178/361] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:07:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:07:59.198117 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f447ac59-f782-40c6-ac78-cc48a5a2bb80 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:59.219148 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f354870-8b40-40a0-b88b-5d018a4d908c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:59.219358 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f354870-8b40-40a0-b88b-5d018a4d908c None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:59.219554 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f354870-8b40-40a0-b88b-5d018a4d908c None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:59.219889 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f354870-8b40-40a0-b88b-5d018a4d908c None admin] PATH_INFO: `/v3/projects/7d6be68cf8b046d5b03fba3c5a367c11` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:59.221058 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f354870-8b40-40a0-b88b-5d018a4d908c None admin] RBAC: Authorizing `identity:get_project(project_id=7d6be68cf8b046d5b03fba3c5a367c11)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:59.222511 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f354870-8b40-40a0-b88b-5d018a4d908c None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:59.224443 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 184/362] 10.222.0.16 () {56 vars in 1186 bytes} [Wed Aug 24 09:07:59 2022] GET /identity/v3/projects/7d6be68cf8b046d5b03fba3c5a367c11 => generated 298 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:07:59.227730 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f354870-8b40-40a0-b88b-5d018a4d908c None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:59.334263 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f447ac59-f782-40c6-ac78-cc48a5a2bb80 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:59.362141 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f354870-8b40-40a0-b88b-5d018a4d908c None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:59.379539 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e862c65e-4791-47f8-b3ae-d1a2b23e01d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:59.380368 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e862c65e-4791-47f8-b3ae-d1a2b23e01d2 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:59.380646 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e862c65e-4791-47f8-b3ae-d1a2b23e01d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:59.380921 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e862c65e-4791-47f8-b3ae-d1a2b23e01d2 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:59.395512 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e862c65e-4791-47f8-b3ae-d1a2b23e01d2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:07:59.401012 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-e862c65e-4791-47f8-b3ae-d1a2b23e01d2 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 09:07:59.401012 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:07:59.401012 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:07:59.401012 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:07:59.401012 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:07:59.401012 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:07:59.401012 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:07:59.401012 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:07:59.401012 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:07:59.401012 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:07:59.401012 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:07:59.401012 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:07:59.401012 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:07:59.401012 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:07:59.401012 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:07:59.401012 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:07:59.401890 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:07:59.401890 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:07:59.401890 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:07:59.401890 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:07:59.401890 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:07:59.401890 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:07:59.401890 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:07:59.401890 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:07:59.401890 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:07:59.401890 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:07:59.401890 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:07:59.401890 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Aug 24 09:07:59.401890 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:07:59.401890 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:07:59.401890 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:07:59.401890 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:07:59.403788 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:07:59.403788 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:07:59.403788 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:07:59.403788 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:07:59.403788 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:07:59.403788 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:07:59.403788 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:07:59.403788 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:07:59.403788 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:07:59.403788 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:07:59.403788 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 09:07:59.403788 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Aug 24 09:07:59.403788 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 179/363] 10.222.0.16 () {56 vars in 1114 bytes} [Wed Aug 24 09:07:59 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:07:59.404703 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836a0a29-6f93-4ab8-929c-6d4bbaf2f365 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:59.404703 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836a0a29-6f93-4ab8-929c-6d4bbaf2f365 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:59.404703 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836a0a29-6f93-4ab8-929c-6d4bbaf2f365 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:59.408188 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf970585-bf65-4f9d-b597-d881dafaf965 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:59.408855 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf970585-bf65-4f9d-b597-d881dafaf965 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:59.409955 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf970585-bf65-4f9d-b597-d881dafaf965 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:59.409955 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf970585-bf65-4f9d-b597-d881dafaf965 None admin] PATH_INFO: `/v3/projects/99f7f068dc714797ac5d88928535f7a2` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:59.410662 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf970585-bf65-4f9d-b597-d881dafaf965 None admin] RBAC: Authorizing `identity:get_project(project_id=99f7f068dc714797ac5d88928535f7a2)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:59.412374 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf970585-bf65-4f9d-b597-d881dafaf965 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:59.414408 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 185/364] 10.222.0.16 () {56 vars in 1186 bytes} [Wed Aug 24 09:07:59 2022] GET /identity/v3/projects/99f7f068dc714797ac5d88928535f7a2 => generated 302 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:07:59.415773 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf970585-bf65-4f9d-b597-d881dafaf965 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:59.429062 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-836a0a29-6f93-4ab8-929c-6d4bbaf2f365 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:07:59.430409 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-836a0a29-6f93-4ab8-929c-6d4bbaf2f365 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:07:59.439334 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:59.453084 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:59.514196 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 180/365] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:07:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:07:59.515457 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836a0a29-6f93-4ab8-929c-6d4bbaf2f365 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:59.549347 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf970585-bf65-4f9d-b597-d881dafaf965 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:59.588268 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53520bf7-dbee-401c-9294-c10ec06b1bbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:59.589317 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53520bf7-dbee-401c-9294-c10ec06b1bbe None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:59.589830 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53520bf7-dbee-401c-9294-c10ec06b1bbe None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:59.590298 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53520bf7-dbee-401c-9294-c10ec06b1bbe None admin] PATH_INFO: `/v3/groups/28f5845ddc6847b38fd62490f9dce3d9` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:59.592215 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53520bf7-dbee-401c-9294-c10ec06b1bbe None admin] RBAC: Authorizing `identity:get_group(group_id=28f5845ddc6847b38fd62490f9dce3d9)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:59.593944 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53520bf7-dbee-401c-9294-c10ec06b1bbe None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:59.596688 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 186/366] 10.222.0.16 () {56 vars in 1180 bytes} [Wed Aug 24 09:07:59 2022] GET /identity/v3/groups/28f5845ddc6847b38fd62490f9dce3d9 => generated 226 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:07:59.597752 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53520bf7-dbee-401c-9294-c10ec06b1bbe None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:59.663960 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836a0a29-6f93-4ab8-929c-6d4bbaf2f365 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:59.718075 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f946541-f80b-439f-a6a2-6ffedb738ad9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:59.718755 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f946541-f80b-439f-a6a2-6ffedb738ad9 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:59.719227 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f946541-f80b-439f-a6a2-6ffedb738ad9 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:59.719508 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f946541-f80b-439f-a6a2-6ffedb738ad9 None admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:59.720234 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f946541-f80b-439f-a6a2-6ffedb738ad9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:07:59.720528 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f946541-f80b-439f-a6a2-6ffedb738ad9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:59.721935 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f946541-f80b-439f-a6a2-6ffedb738ad9 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:59.727228 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:59.734704 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 181/367] 10.222.0.16 () {56 vars in 1111 bytes} [Wed Aug 24 09:07:59 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:07:59.735369 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f946541-f80b-439f-a6a2-6ffedb738ad9 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:59.740539 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53520bf7-dbee-401c-9294-c10ec06b1bbe None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:59.743110 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:07:59.780901 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a79d6a8-d3ba-4930-b558-4db21e1a0256 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:59.781934 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a79d6a8-d3ba-4930-b558-4db21e1a0256 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:59.782405 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a79d6a8-d3ba-4930-b558-4db21e1a0256 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:59.782852 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a79d6a8-d3ba-4930-b558-4db21e1a0256 None admin] PATH_INFO: `/v3/groups/28f5845ddc6847b38fd62490f9dce3d9` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:59.784282 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a79d6a8-d3ba-4930-b558-4db21e1a0256 None admin] RBAC: Authorizing `identity:get_group(group_id=28f5845ddc6847b38fd62490f9dce3d9)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:59.785997 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a79d6a8-d3ba-4930-b558-4db21e1a0256 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:59.787932 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 187/368] 10.222.0.16 () {56 vars in 1180 bytes} [Wed Aug 24 09:07:59 2022] GET /identity/v3/groups/28f5845ddc6847b38fd62490f9dce3d9 => generated 226 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:07:59.789002 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a79d6a8-d3ba-4930-b558-4db21e1a0256 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:59.865807 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f946541-f80b-439f-a6a2-6ffedb738ad9 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:59.907488 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59e175c-c63b-4cbf-ab82-a6e7ca793991 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:07:59.908133 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59e175c-c63b-4cbf-ab82-a6e7ca793991 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:07:59.908418 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59e175c-c63b-4cbf-ab82-a6e7ca793991 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:07:59.909931 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59e175c-c63b-4cbf-ab82-a6e7ca793991 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:07:59.909931 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59e175c-c63b-4cbf-ab82-a6e7ca793991 None admin] RBAC: Adding query filter params (group.id=28f5845ddc6847b38fd62490f9dce3d9, role.id=ed6466d17fb7403c8491bb7f1ef95fc1, scope.project.id=7d6be68cf8b046d5b03fba3c5a367c11) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:07:59.909931 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59e175c-c63b-4cbf-ab82-a6e7ca793991 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:07:59.911754 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59e175c-c63b-4cbf-ab82-a6e7ca793991 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:07:59.917122 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 182/369] 10.222.0.16 () {56 vars in 1376 bytes} [Wed Aug 24 09:07:59 2022] GET /identity/v3/role_assignments?group.id=28f5845ddc6847b38fd62490f9dce3d9&scope.project.id=7d6be68cf8b046d5b03fba3c5a367c11&role.id=ed6466d17fb7403c8491bb7f1ef95fc1 => generated 260 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:07:59.917733 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59e175c-c63b-4cbf-ab82-a6e7ca793991 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:07:59.941189 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a79d6a8-d3ba-4930-b558-4db21e1a0256 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:07:59.999089 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252b13b9-2ca9-4967-acd6-765dcf751ed8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:00.000548 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252b13b9-2ca9-4967-acd6-765dcf751ed8 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:00.001346 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252b13b9-2ca9-4967-acd6-765dcf751ed8 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:00.001837 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252b13b9-2ca9-4967-acd6-765dcf751ed8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:00.012391 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:00.041904 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:00.064283 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59e175c-c63b-4cbf-ab82-a6e7ca793991 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:00.101528 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252b13b9-2ca9-4967-acd6-765dcf751ed8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:00.103081 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252b13b9-2ca9-4967-acd6-765dcf751ed8 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:00.108735 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588e34d3-8413-401a-9cac-4899c971abd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:00.109416 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588e34d3-8413-401a-9cac-4899c971abd7 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:00.109706 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588e34d3-8413-401a-9cac-4899c971abd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:00.109982 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588e34d3-8413-401a-9cac-4899c971abd7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:00.110741 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588e34d3-8413-401a-9cac-4899c971abd7 None admin] RBAC: Adding query filter params (group.id=28f5845ddc6847b38fd62490f9dce3d9, role.id=136cdf78e00a426997dd4e37faff489b, scope.project.id=99f7f068dc714797ac5d88928535f7a2) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:00.111088 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588e34d3-8413-401a-9cac-4899c971abd7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:00.112778 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588e34d3-8413-401a-9cac-4899c971abd7 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:00.119061 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 183/370] 10.222.0.16 () {56 vars in 1376 bytes} [Wed Aug 24 09:07:59 2022] GET /identity/v3/role_assignments?group.id=28f5845ddc6847b38fd62490f9dce3d9&scope.project.id=99f7f068dc714797ac5d88928535f7a2&role.id=136cdf78e00a426997dd4e37faff489b => generated 260 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:00.119846 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588e34d3-8413-401a-9cac-4899c971abd7 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:00.254345 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588e34d3-8413-401a-9cac-4899c971abd7 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:00.292046 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:00.298677 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7027170b-4343-4438-bc7c-183e3c526824 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:00.299344 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7027170b-4343-4438-bc7c-183e3c526824 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:00.299641 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7027170b-4343-4438-bc7c-183e3c526824 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:00.299923 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7027170b-4343-4438-bc7c-183e3c526824 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:00.317217 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7027170b-4343-4438-bc7c-183e3c526824 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:08:00.322605 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-7027170b-4343-4438-bc7c-183e3c526824 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:08:00.322605 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:08:00.322605 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:08:00.322605 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:08:00.322605 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:08:00.322605 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:08:00.322605 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:08:00.322605 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:08:00.322605 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:08:00.322605 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:08:00.322605 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:08:00.322605 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:08:00.322605 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:08:00.322605 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:08:00.322605 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:08:00.322605 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:08:00.323671 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:08:00.323671 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:08:00.323671 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:08:00.323671 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:08:00.323671 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:08:00.323671 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:08:00.323671 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:08:00.323671 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:08:00.323671 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:08:00.323671 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:08:00.323671 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:08:00.323671 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Aug 24 09:08:00.323671 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:08:00.323671 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:08:00.323671 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:08:00.323671 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:08:00.324685 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:08:00.324685 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:08:00.324685 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:08:00.324685 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:08:00.324685 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:08:00.324685 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:08:00.324685 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:08:00.324685 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:08:00.324685 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:08:00.324685 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:08:00.324685 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:08:00.324685 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Aug 24 09:08:00.325502 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 188/371] 10.222.0.16 () {58 vars in 1282 bytes} [Wed Aug 24 09:07:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 534 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 09:08:00.325594 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 184/372] 10.222.0.16 () {56 vars in 1096 bytes} [Wed Aug 24 09:08:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:08:00.329359 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252b13b9-2ca9-4967-acd6-765dcf751ed8 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:00.332372 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7027170b-4343-4438-bc7c-183e3c526824 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:00.417753 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:00.458665 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7027170b-4343-4438-bc7c-183e3c526824 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:00.495690 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35996fc7-2a8f-40ba-94b2-b8e029ffca7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:00.496254 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35996fc7-2a8f-40ba-94b2-b8e029ffca7d None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:00.496540 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35996fc7-2a8f-40ba-94b2-b8e029ffca7d None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:00.496850 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35996fc7-2a8f-40ba-94b2-b8e029ffca7d None admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:00.497156 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252b13b9-2ca9-4967-acd6-765dcf751ed8 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:00.497522 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35996fc7-2a8f-40ba-94b2-b8e029ffca7d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:00.497828 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35996fc7-2a8f-40ba-94b2-b8e029ffca7d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:00.499200 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35996fc7-2a8f-40ba-94b2-b8e029ffca7d None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:00.509197 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 185/373] 10.222.0.16 () {56 vars in 1099 bytes} [Wed Aug 24 09:08:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:00.514395 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35996fc7-2a8f-40ba-94b2-b8e029ffca7d None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:00.542872 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663a7048-221f-4fe8-910c-e3666d88fd70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:00.544442 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663a7048-221f-4fe8-910c-e3666d88fd70 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:00.544894 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663a7048-221f-4fe8-910c-e3666d88fd70 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:00.545309 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663a7048-221f-4fe8-910c-e3666d88fd70 None admin] PATH_INFO: `/v3/projects/99f7f068dc714797ac5d88928535f7a2` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:00.546694 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663a7048-221f-4fe8-910c-e3666d88fd70 None admin] RBAC: Authorizing `identity:get_project(project_id=99f7f068dc714797ac5d88928535f7a2)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:00.548544 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663a7048-221f-4fe8-910c-e3666d88fd70 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:00.550683 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 189/374] 10.222.0.16 () {56 vars in 1186 bytes} [Wed Aug 24 09:08:00 2022] GET /identity/v3/projects/99f7f068dc714797ac5d88928535f7a2 => generated 302 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:08:00.557963 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663a7048-221f-4fe8-910c-e3666d88fd70 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:00.595895 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:00.643477 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35996fc7-2a8f-40ba-94b2-b8e029ffca7d None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:00.680224 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c3d30e-cd5b-476d-a147-dcad0b490665 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:00.680857 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c3d30e-cd5b-476d-a147-dcad0b490665 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:00.681207 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c3d30e-cd5b-476d-a147-dcad0b490665 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:00.681551 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c3d30e-cd5b-476d-a147-dcad0b490665 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:00.754132 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c3d30e-cd5b-476d-a147-dcad0b490665 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:00.755903 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c3d30e-cd5b-476d-a147-dcad0b490665 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:00.779698 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663a7048-221f-4fe8-910c-e3666d88fd70 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:00.815058 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:00.829859 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:00.843610 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d589d94-2399-4c58-8383-ec787906d5cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:00.844211 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d589d94-2399-4c58-8383-ec787906d5cf None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:00.844493 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d589d94-2399-4c58-8383-ec787906d5cf None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:00.844768 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d589d94-2399-4c58-8383-ec787906d5cf None admin] PATH_INFO: `/v3/groups/28f5845ddc6847b38fd62490f9dce3d9` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:00.845896 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d589d94-2399-4c58-8383-ec787906d5cf None admin] RBAC: Authorizing `identity:get_group(group_id=28f5845ddc6847b38fd62490f9dce3d9)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:00.847342 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d589d94-2399-4c58-8383-ec787906d5cf None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:00.849327 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 190/375] 10.222.0.16 () {56 vars in 1180 bytes} [Wed Aug 24 09:08:00 2022] GET /identity/v3/groups/28f5845ddc6847b38fd62490f9dce3d9 => generated 226 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:00.860744 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d589d94-2399-4c58-8383-ec787906d5cf None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:00.872660 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 186/376] 10.222.0.16 () {58 vars in 1282 bytes} [Wed Aug 24 09:08:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 09:08:00.878407 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c3d30e-cd5b-476d-a147-dcad0b490665 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:01.004255 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c3d30e-cd5b-476d-a147-dcad0b490665 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:01.047162 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d589d94-2399-4c58-8383-ec787906d5cf None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:01.049425 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8faa4e8f-57be-40fc-b1d4-ac3e6b54d832 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:01.050264 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8faa4e8f-57be-40fc-b1d4-ac3e6b54d832 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:01.050742 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8faa4e8f-57be-40fc-b1d4-ac3e6b54d832 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:01.051189 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8faa4e8f-57be-40fc-b1d4-ac3e6b54d832 None admin] PATH_INFO: `/v3/projects/e359feabfca34b0e8852e124e6b4b5a1` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:01.052595 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8faa4e8f-57be-40fc-b1d4-ac3e6b54d832 None admin] RBAC: Authorizing `identity:get_project(project_id=e359feabfca34b0e8852e124e6b4b5a1)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:01.054243 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8faa4e8f-57be-40fc-b1d4-ac3e6b54d832 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:01.057508 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 187/377] 10.222.0.16 () {56 vars in 1186 bytes} [Wed Aug 24 09:08:00 2022] GET /identity/v3/projects/e359feabfca34b0e8852e124e6b4b5a1 => generated 344 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:01.062110 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8faa4e8f-57be-40fc-b1d4-ac3e6b54d832 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:01.111915 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:01.112886 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19106d24-ef5f-4db0-bbc4-9670faa6b60f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:01.113510 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19106d24-ef5f-4db0-bbc4-9670faa6b60f None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:01.113788 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19106d24-ef5f-4db0-bbc4-9670faa6b60f None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:01.114062 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19106d24-ef5f-4db0-bbc4-9670faa6b60f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:01.114792 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19106d24-ef5f-4db0-bbc4-9670faa6b60f None admin] RBAC: Adding query filter params (group.id=28f5845ddc6847b38fd62490f9dce3d9, role.id=ed6466d17fb7403c8491bb7f1ef95fc1, scope.project.id=99f7f068dc714797ac5d88928535f7a2) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:01.115104 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19106d24-ef5f-4db0-bbc4-9670faa6b60f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:01.116563 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19106d24-ef5f-4db0-bbc4-9670faa6b60f None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:01.129946 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 191/378] 10.222.0.16 () {56 vars in 1376 bytes} [Wed Aug 24 09:08:00 2022] GET /identity/v3/role_assignments?group.id=28f5845ddc6847b38fd62490f9dce3d9&scope.project.id=99f7f068dc714797ac5d88928535f7a2&role.id=ed6466d17fb7403c8491bb7f1ef95fc1 => generated 260 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:08:01.229223 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8faa4e8f-57be-40fc-b1d4-ac3e6b54d832 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:01.271912 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc83da75-85a4-4381-a41a-6aebf8329f8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:01.272480 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc83da75-85a4-4381-a41a-6aebf8329f8c None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:01.272754 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc83da75-85a4-4381-a41a-6aebf8329f8c None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:01.273125 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc83da75-85a4-4381-a41a-6aebf8329f8c None admin] PATH_INFO: `/v3/groups/9876c820bdd643c2bae408d77ffba313` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:01.280433 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc83da75-85a4-4381-a41a-6aebf8329f8c None admin] RBAC: Authorizing `identity:get_group(group_id=9876c820bdd643c2bae408d77ffba313)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:01.281922 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc83da75-85a4-4381-a41a-6aebf8329f8c None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:01.283637 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 188/379] 10.222.0.16 () {56 vars in 1180 bytes} [Wed Aug 24 09:08:01 2022] GET /identity/v3/groups/9876c820bdd643c2bae408d77ffba313 => generated 229 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:01.289548 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19106d24-ef5f-4db0-bbc4-9670faa6b60f None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:01.362628 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:01.421360 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19106d24-ef5f-4db0-bbc4-9670faa6b60f None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:01.472737 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32429773-1d52-4f09-8d6f-084dbee6d197 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:01.474206 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32429773-1d52-4f09-8d6f-084dbee6d197 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:01.474992 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32429773-1d52-4f09-8d6f-084dbee6d197 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:01.477396 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 189/380] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:08:01 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:08:01.485322 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b16403c-4a99-4a60-b9a8-35257928ec85 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:01.485322 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b16403c-4a99-4a60-b9a8-35257928ec85 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:01.485322 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b16403c-4a99-4a60-b9a8-35257928ec85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:01.495607 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70833ba-bf81-473f-b288-c14cdfb78b35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:01.495607 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70833ba-bf81-473f-b288-c14cdfb78b35 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:01.495607 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70833ba-bf81-473f-b288-c14cdfb78b35 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:01.495607 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70833ba-bf81-473f-b288-c14cdfb78b35 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:01.495981 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70833ba-bf81-473f-b288-c14cdfb78b35 None admin] RBAC: Adding query filter params (group.id=9876c820bdd643c2bae408d77ffba313, role.id=15f634d4a2d247ddab90c458e5e1c141, scope.project.id=e359feabfca34b0e8852e124e6b4b5a1) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:01.495981 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70833ba-bf81-473f-b288-c14cdfb78b35 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:01.498506 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70833ba-bf81-473f-b288-c14cdfb78b35 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:01.507616 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 192/381] 10.222.0.16 () {56 vars in 1376 bytes} [Wed Aug 24 09:08:01 2022] GET /identity/v3/role_assignments?group.id=9876c820bdd643c2bae408d77ffba313&scope.project.id=e359feabfca34b0e8852e124e6b4b5a1&role.id=15f634d4a2d247ddab90c458e5e1c141 => generated 260 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:01.532802 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-2b16403c-4a99-4a60-b9a8-35257928ec85 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:08:01.534076 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-2b16403c-4a99-4a60-b9a8-35257928ec85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:01.646303 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 190/382] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:08:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:08:02.085468 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70833ba-bf81-473f-b288-c14cdfb78b35 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:02.202921 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70833ba-bf81-473f-b288-c14cdfb78b35 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:02.230640 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:02.245524 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed219a23-c61a-4725-b159-5ae8e583ddf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:02.246286 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed219a23-c61a-4725-b159-5ae8e583ddf6 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:02.256319 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed219a23-c61a-4725-b159-5ae8e583ddf6 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:02.256319 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed219a23-c61a-4725-b159-5ae8e583ddf6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:02.258910 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed219a23-c61a-4725-b159-5ae8e583ddf6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:08:02.264531 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-ed219a23-c61a-4725-b159-5ae8e583ddf6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 09:08:02.264531 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:08:02.264531 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:08:02.264531 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:08:02.264531 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:08:02.264531 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:08:02.264531 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:08:02.264531 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:08:02.264531 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:08:02.264531 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:08:02.264531 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:08:02.264531 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:08:02.264531 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:08:02.264531 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:08:02.264531 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:08:02.264531 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:08:02.265702 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:08:02.265702 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:08:02.265702 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:08:02.265702 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:08:02.265702 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:08:02.265702 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:08:02.265702 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:08:02.265702 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:08:02.265702 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:08:02.265702 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:08:02.265702 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:08:02.265702 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 24 09:08:02.265702 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:08:02.265702 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:08:02.265702 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:08:02.265702 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:08:02.266684 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:08:02.266684 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:08:02.266684 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:08:02.266684 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:08:02.266684 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:08:02.266684 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:08:02.266684 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:08:02.266684 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:08:02.266684 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:08:02.266684 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:08:02.266684 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 09:08:02.266684 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 24 09:08:02.266684 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 193/383] 10.222.0.16 () {56 vars in 1099 bytes} [Wed Aug 24 09:08:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:08:02.273366 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b16403c-4a99-4a60-b9a8-35257928ec85 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:02.304286 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:02.397648 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b16403c-4a99-4a60-b9a8-35257928ec85 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:02.435300 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0652b965-dd7b-4f9c-a548-cd484509d3c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:02.435966 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0652b965-dd7b-4f9c-a548-cd484509d3c1 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:02.436246 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0652b965-dd7b-4f9c-a548-cd484509d3c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:02.436519 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0652b965-dd7b-4f9c-a548-cd484509d3c1 None admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:02.437193 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0652b965-dd7b-4f9c-a548-cd484509d3c1 None admin] RBAC: Adding query filter params (name=member) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:02.437498 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0652b965-dd7b-4f9c-a548-cd484509d3c1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:02.438872 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0652b965-dd7b-4f9c-a548-cd484509d3c1 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:02.442482 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8fc48b-5ec4-47e0-bd03-6774c96b12ab None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:02.442927 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8fc48b-5ec4-47e0-bd03-6774c96b12ab None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:02.443001 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8fc48b-5ec4-47e0-bd03-6774c96b12ab None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:02.449349 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 194/384] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:08:02 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 09:08:02.450628 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cd5abb-f8fe-445d-ba37-06437c174cf6 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:02.450911 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cd5abb-f8fe-445d-ba37-06437c174cf6 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:02.451188 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cd5abb-f8fe-445d-ba37-06437c174cf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:02.453888 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 191/385] 10.222.0.16 () {56 vars in 1101 bytes} [Wed Aug 24 09:08:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:02.460858 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0652b965-dd7b-4f9c-a548-cd484509d3c1 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:02.471927 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-a5cd5abb-f8fe-445d-ba37-06437c174cf6 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:08:02.473283 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-a5cd5abb-f8fe-445d-ba37-06437c174cf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:02.562104 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 195/386] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:08:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:08:02.594243 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:02.616879 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0652b965-dd7b-4f9c-a548-cd484509d3c1 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:02.653881 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac317b6-2618-4396-a389-c05250080a30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:02.654863 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac317b6-2618-4396-a389-c05250080a30 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:02.655343 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac317b6-2618-4396-a389-c05250080a30 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:02.655841 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac317b6-2618-4396-a389-c05250080a30 None admin] PATH_INFO: `/v3/groups/28f5845ddc6847b38fd62490f9dce3d9` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:02.657169 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac317b6-2618-4396-a389-c05250080a30 None admin] RBAC: Authorizing `identity:get_group(group_id=28f5845ddc6847b38fd62490f9dce3d9)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:02.658803 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac317b6-2618-4396-a389-c05250080a30 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:02.660926 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 192/387] 10.222.0.16 () {56 vars in 1180 bytes} [Wed Aug 24 09:08:02 2022] GET /identity/v3/groups/28f5845ddc6847b38fd62490f9dce3d9 => generated 226 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:08:02.666664 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cd5abb-f8fe-445d-ba37-06437c174cf6 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:02.771794 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cd5abb-f8fe-445d-ba37-06437c174cf6 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:02.787554 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:02.810092 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14037465-7cf0-488f-ab78-2bf4c2855924 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:02.811091 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14037465-7cf0-488f-ab78-2bf4c2855924 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:02.811605 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14037465-7cf0-488f-ab78-2bf4c2855924 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:02.812130 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14037465-7cf0-488f-ab78-2bf4c2855924 None admin] PATH_INFO: `/v3/projects/7d6be68cf8b046d5b03fba3c5a367c11` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:02.813485 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14037465-7cf0-488f-ab78-2bf4c2855924 None admin] RBAC: Authorizing `identity:get_project(project_id=7d6be68cf8b046d5b03fba3c5a367c11)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:02.815132 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14037465-7cf0-488f-ab78-2bf4c2855924 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:02.818015 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 196/388] 10.222.0.16 () {56 vars in 1186 bytes} [Wed Aug 24 09:08:02 2022] GET /identity/v3/projects/7d6be68cf8b046d5b03fba3c5a367c11 => generated 298 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:02.823868 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac317b6-2618-4396-a389-c05250080a30 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:02.944445 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac317b6-2618-4396-a389-c05250080a30 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:02.968081 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7a149e-455f-42c9-a66e-bc7e5e030571 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:02.968694 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7a149e-455f-42c9-a66e-bc7e5e030571 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:02.968971 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7a149e-455f-42c9-a66e-bc7e5e030571 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:02.970442 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 197/389] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:08:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:08:02.977645 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a7dd95-0b28-4fbc-9961-bcd962905720 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:02.977923 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a7dd95-0b28-4fbc-9961-bcd962905720 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:02.978192 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a7dd95-0b28-4fbc-9961-bcd962905720 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:02.996291 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229aa19c-47a3-44cf-b71f-12dd49f3d935 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:02.998018 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229aa19c-47a3-44cf-b71f-12dd49f3d935 None admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:02.998504 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229aa19c-47a3-44cf-b71f-12dd49f3d935 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:02.999042 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229aa19c-47a3-44cf-b71f-12dd49f3d935 None admin] PATH_INFO: `/v3/projects/7d6be68cf8b046d5b03fba3c5a367c11/groups/28f5845ddc6847b38fd62490f9dce3d9/roles/136cdf78e00a426997dd4e37faff489b` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:03.001267 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229aa19c-47a3-44cf-b71f-12dd49f3d935 None admin] RBAC: Authorizing `identity:create_grant(project_id=7d6be68cf8b046d5b03fba3c5a367c11, group_id=28f5845ddc6847b38fd62490f9dce3d9, role_id=136cdf78e00a426997dd4e37faff489b)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:03.002943 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229aa19c-47a3-44cf-b71f-12dd49f3d935 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:03.005471 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-f6a7dd95-0b28-4fbc-9961-bcd962905720 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:08:03.006850 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-f6a7dd95-0b28-4fbc-9961-bcd962905720 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:03.077582 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 198/390] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:08:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 09:08:03.094432 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 193/391] 10.222.0.16 () {58 vars in 1442 bytes} [Wed Aug 24 09:08:02 2022] PUT /identity/v3/projects/7d6be68cf8b046d5b03fba3c5a367c11/groups/28f5845ddc6847b38fd62490f9dce3d9/roles/136cdf78e00a426997dd4e37faff489b => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:08:03.118393 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a7dd95-0b28-4fbc-9961-bcd962905720 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:03.251098 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a7dd95-0b28-4fbc-9961-bcd962905720 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:03.269711 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:03.298144 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b4c26f-df7b-47db-8d52-c03c81ab2bfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:03.298790 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b4c26f-df7b-47db-8d52-c03c81ab2bfb None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:03.299071 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b4c26f-df7b-47db-8d52-c03c81ab2bfb None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:03.299349 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b4c26f-df7b-47db-8d52-c03c81ab2bfb None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:03.324591 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b4c26f-df7b-47db-8d52-c03c81ab2bfb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:08:03.330186 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-89b4c26f-df7b-47db-8d52-c03c81ab2bfb None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 09:08:03.330186 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:08:03.330186 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:08:03.330186 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:08:03.330186 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:08:03.330186 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:08:03.330186 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:08:03.330186 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:08:03.330186 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:08:03.330186 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:08:03.330186 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:08:03.330186 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:08:03.330186 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:08:03.330186 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:08:03.330186 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:08:03.330186 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:08:03.331073 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:08:03.331073 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:08:03.331073 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:08:03.331073 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:08:03.331073 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:08:03.331073 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:08:03.331073 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:08:03.331073 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:08:03.331073 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:08:03.331073 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:08:03.331073 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:08:03.331073 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 24 09:08:03.331073 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:08:03.331073 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:08:03.331073 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:08:03.331073 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:08:03.331899 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:08:03.331899 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:08:03.331899 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:08:03.331899 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:08:03.331899 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:08:03.331899 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:08:03.331899 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:08:03.331899 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:08:03.331899 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:08:03.331899 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:08:03.331899 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 09:08:03.331899 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 24 09:08:03.331899 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 199/392] 10.222.0.16 () {56 vars in 1114 bytes} [Wed Aug 24 09:08:03 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:08:03.337396 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229aa19c-47a3-44cf-b71f-12dd49f3d935 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:03.357639 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:03.409557 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be78cb9-25e5-4d32-b61f-60ee4d2f4a13 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:03.409557 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be78cb9-25e5-4d32-b61f-60ee4d2f4a13 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:03.409826 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be78cb9-25e5-4d32-b61f-60ee4d2f4a13 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:03.412925 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 200/393] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:08:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:08:03.416740 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db7c479-849e-448d-a53a-8662e7f77a93 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:03.417022 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db7c479-849e-448d-a53a-8662e7f77a93 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:03.417471 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db7c479-849e-448d-a53a-8662e7f77a93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:03.439525 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-0db7c479-849e-448d-a53a-8662e7f77a93 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:08:03.441915 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-0db7c479-849e-448d-a53a-8662e7f77a93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:03.504867 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229aa19c-47a3-44cf-b71f-12dd49f3d935 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:03.581791 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 201/394] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:08:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 09:08:03.583384 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db7c479-849e-448d-a53a-8662e7f77a93 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:03.585200 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6f30cf-5b22-4b63-8c1d-f5ab7525b366 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:03.586523 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6f30cf-5b22-4b63-8c1d-f5ab7525b366 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:03.596068 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6f30cf-5b22-4b63-8c1d-f5ab7525b366 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:03.597015 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6f30cf-5b22-4b63-8c1d-f5ab7525b366 None admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:03.597935 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6f30cf-5b22-4b63-8c1d-f5ab7525b366 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:03.598714 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6f30cf-5b22-4b63-8c1d-f5ab7525b366 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:03.601070 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6f30cf-5b22-4b63-8c1d-f5ab7525b366 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:03.642081 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 194/395] 10.222.0.16 () {56 vars in 1111 bytes} [Wed Aug 24 09:08:03 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:03.643443 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6f30cf-5b22-4b63-8c1d-f5ab7525b366 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:03.859393 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db7c479-849e-448d-a53a-8662e7f77a93 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:03.863017 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:03.866438 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:03.911269 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bac08c-5d5d-4382-bfc6-ad2bb658a195 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:03.912717 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bac08c-5d5d-4382-bfc6-ad2bb658a195 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:03.913004 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bac08c-5d5d-4382-bfc6-ad2bb658a195 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:03.914205 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bac08c-5d5d-4382-bfc6-ad2bb658a195 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:03.931651 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bac08c-5d5d-4382-bfc6-ad2bb658a195 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:08:03.936622 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-97bac08c-5d5d-4382-bfc6-ad2bb658a195 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 09:08:03.936622 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:08:03.936622 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:08:03.936622 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:08:03.936622 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:08:03.936622 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:08:03.936622 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:08:03.936622 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:08:03.936622 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:08:03.936622 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:08:03.936622 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:08:03.936622 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:08:03.936622 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:08:03.936622 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:08:03.936622 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:08:03.936622 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:08:03.937507 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:08:03.937507 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:08:03.937507 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:08:03.937507 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:08:03.937507 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:08:03.937507 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:08:03.937507 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:08:03.937507 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:08:03.937507 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:08:03.937507 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:08:03.937507 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:08:03.937507 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 24 09:08:03.937507 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:08:03.937507 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:08:03.937507 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:08:03.937507 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:08:03.938849 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:08:03.938849 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:08:03.938849 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:08:03.938849 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:08:03.938849 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:08:03.938849 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:08:03.938849 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:08:03.938849 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:08:03.938849 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:08:03.938849 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:08:03.938849 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 09:08:03.938849 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 24 09:08:03.938849 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 202/396] 10.222.0.16 () {56 vars in 1099 bytes} [Wed Aug 24 09:08:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 355 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:08:03.943402 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bac08c-5d5d-4382-bfc6-ad2bb658a195 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:03.963115 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6f30cf-5b22-4b63-8c1d-f5ab7525b366 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:04.013647 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5855c4b-5fbc-4544-a7a4-04c83683b858 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:04.014500 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5855c4b-5fbc-4544-a7a4-04c83683b858 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:04.014784 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5855c4b-5fbc-4544-a7a4-04c83683b858 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:04.015060 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5855c4b-5fbc-4544-a7a4-04c83683b858 None admin] PATH_INFO: `/v3/groups/28f5845ddc6847b38fd62490f9dce3d9` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:04.016405 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5855c4b-5fbc-4544-a7a4-04c83683b858 None admin] RBAC: Authorizing `identity:get_group(group_id=28f5845ddc6847b38fd62490f9dce3d9)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:04.018123 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5855c4b-5fbc-4544-a7a4-04c83683b858 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:04.020200 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 195/397] 10.222.0.16 () {56 vars in 1180 bytes} [Wed Aug 24 09:08:03 2022] GET /identity/v3/groups/28f5845ddc6847b38fd62490f9dce3d9 => generated 226 bytes in 377 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:04.021128 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5855c4b-5fbc-4544-a7a4-04c83683b858 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:04.072566 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bac08c-5d5d-4382-bfc6-ad2bb658a195 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:04.107790 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d89afc-cd1c-4705-95a9-1fada45c7175 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:04.108397 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d89afc-cd1c-4705-95a9-1fada45c7175 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:04.108674 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d89afc-cd1c-4705-95a9-1fada45c7175 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:04.108946 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d89afc-cd1c-4705-95a9-1fada45c7175 None admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:04.109664 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d89afc-cd1c-4705-95a9-1fada45c7175 None admin] RBAC: Adding query filter params (name=member) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:04.109910 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d89afc-cd1c-4705-95a9-1fada45c7175 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:04.111289 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d89afc-cd1c-4705-95a9-1fada45c7175 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:04.119349 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 203/398] 10.222.0.16 () {56 vars in 1101 bytes} [Wed Aug 24 09:08:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:04.120160 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d89afc-cd1c-4705-95a9-1fada45c7175 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:04.136135 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5855c4b-5fbc-4544-a7a4-04c83683b858 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:04.151022 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:04.170232 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ee2555-f4ea-4f55-b099-4d9fcca25747 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:04.170896 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ee2555-f4ea-4f55-b099-4d9fcca25747 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:04.171179 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ee2555-f4ea-4f55-b099-4d9fcca25747 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:04.171453 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ee2555-f4ea-4f55-b099-4d9fcca25747 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:04.174456 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:04.185485 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ee2555-f4ea-4f55-b099-4d9fcca25747 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:08:04.190530 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-e0ee2555-f4ea-4f55-b099-4d9fcca25747 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 09:08:04.190530 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:08:04.190530 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:08:04.190530 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:08:04.190530 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:08:04.190530 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:08:04.190530 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:08:04.190530 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:08:04.190530 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:08:04.190530 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:08:04.190530 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:08:04.190530 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:08:04.190530 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:08:04.190530 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:08:04.190530 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:08:04.190530 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:08:04.197037 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:08:04.197037 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:08:04.197037 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:08:04.197037 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:08:04.197037 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:08:04.197037 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:08:04.197037 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:08:04.197037 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:08:04.197037 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:08:04.197037 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:08:04.197037 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:08:04.197037 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Aug 24 09:08:04.197037 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:08:04.197037 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:08:04.197037 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:08:04.197037 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:08:04.197848 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:08:04.197848 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:08:04.197848 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:08:04.197848 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:08:04.197848 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:08:04.197848 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:08:04.197848 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:08:04.197848 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:08:04.197848 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:08:04.197848 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:08:04.197848 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 09:08:04.197848 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Aug 24 09:08:04.197848 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 196/399] 10.222.0.16 () {56 vars in 1114 bytes} [Wed Aug 24 09:08:04 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:08:04.197848 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2214c0-6f12-4deb-80c4-2875dc4ccf22 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:04.198561 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2214c0-6f12-4deb-80c4-2875dc4ccf22 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:04.198561 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2214c0-6f12-4deb-80c4-2875dc4ccf22 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:04.198561 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 197/400] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:08:04 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:08:04.198561 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2214c0-6f12-4deb-80c4-2875dc4ccf22 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:04.248453 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d89afc-cd1c-4705-95a9-1fada45c7175 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:04.283929 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42fd45a-f8c2-4c2c-8fae-4d25a4afef15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:04.284558 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42fd45a-f8c2-4c2c-8fae-4d25a4afef15 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:04.284839 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42fd45a-f8c2-4c2c-8fae-4d25a4afef15 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:04.285116 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42fd45a-f8c2-4c2c-8fae-4d25a4afef15 None admin] PATH_INFO: `/v3/projects/7d6be68cf8b046d5b03fba3c5a367c11` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:04.286249 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42fd45a-f8c2-4c2c-8fae-4d25a4afef15 None admin] RBAC: Authorizing `identity:get_project(project_id=7d6be68cf8b046d5b03fba3c5a367c11)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:04.287834 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42fd45a-f8c2-4c2c-8fae-4d25a4afef15 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:04.294026 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 204/401] 10.222.0.16 () {56 vars in 1186 bytes} [Wed Aug 24 09:08:04 2022] GET /identity/v3/projects/7d6be68cf8b046d5b03fba3c5a367c11 => generated 298 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:04.294026 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42fd45a-f8c2-4c2c-8fae-4d25a4afef15 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:04.315442 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2214c0-6f12-4deb-80c4-2875dc4ccf22 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:04.357480 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd8ee48-7311-4147-b2f9-6bbe389a4b27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:04.358091 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd8ee48-7311-4147-b2f9-6bbe389a4b27 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:04.358377 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd8ee48-7311-4147-b2f9-6bbe389a4b27 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:04.358654 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd8ee48-7311-4147-b2f9-6bbe389a4b27 None admin] PATH_INFO: `/v3/groups/28f5845ddc6847b38fd62490f9dce3d9` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:04.359877 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd8ee48-7311-4147-b2f9-6bbe389a4b27 None admin] RBAC: Authorizing `identity:get_group(group_id=28f5845ddc6847b38fd62490f9dce3d9)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:04.361268 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd8ee48-7311-4147-b2f9-6bbe389a4b27 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:04.363001 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 198/402] 10.222.0.16 () {56 vars in 1180 bytes} [Wed Aug 24 09:08:04 2022] GET /identity/v3/groups/28f5845ddc6847b38fd62490f9dce3d9 => generated 226 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:04.364899 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0fabaf-0ef4-4243-8ce1-aa945142f210 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:04.365182 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0fabaf-0ef4-4243-8ce1-aa945142f210 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:04.365453 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0fabaf-0ef4-4243-8ce1-aa945142f210 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:04.385285 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-ad0fabaf-0ef4-4243-8ce1-aa945142f210 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:08:04.386925 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-ad0fabaf-0ef4-4243-8ce1-aa945142f210 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:04.390223 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:04.399396 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42fd45a-f8c2-4c2c-8fae-4d25a4afef15 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:04.440408 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b9c543-8094-4d49-84e4-e1817e6e174b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:04.441294 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b9c543-8094-4d49-84e4-e1817e6e174b None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:04.441584 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b9c543-8094-4d49-84e4-e1817e6e174b None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:04.441855 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b9c543-8094-4d49-84e4-e1817e6e174b None admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:04.442514 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b9c543-8094-4d49-84e4-e1817e6e174b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:04.442816 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b9c543-8094-4d49-84e4-e1817e6e174b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:04.444889 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b9c543-8094-4d49-84e4-e1817e6e174b None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:04.449103 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:04.455541 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 205/403] 10.222.0.16 () {56 vars in 1111 bytes} [Wed Aug 24 09:08:04 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:04.456364 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b9c543-8094-4d49-84e4-e1817e6e174b None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:04.462884 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 199/404] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:08:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:08:04.464026 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0fabaf-0ef4-4243-8ce1-aa945142f210 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:04.573424 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0fabaf-0ef4-4243-8ce1-aa945142f210 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:04.577958 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b9c543-8094-4d49-84e4-e1817e6e174b None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:04.610899 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6222406-b3be-4a70-bda7-8abf17c5e98d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:04.611522 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6222406-b3be-4a70-bda7-8abf17c5e98d None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:04.611788 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6222406-b3be-4a70-bda7-8abf17c5e98d None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:04.612527 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6222406-b3be-4a70-bda7-8abf17c5e98d None admin] PATH_INFO: `/v3/projects/99f7f068dc714797ac5d88928535f7a2` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:04.613923 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6222406-b3be-4a70-bda7-8abf17c5e98d None admin] RBAC: Authorizing `identity:get_project(project_id=99f7f068dc714797ac5d88928535f7a2)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:04.615363 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6222406-b3be-4a70-bda7-8abf17c5e98d None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:04.622057 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 200/405] 10.222.0.16 () {56 vars in 1186 bytes} [Wed Aug 24 09:08:04 2022] GET /identity/v3/projects/99f7f068dc714797ac5d88928535f7a2 => generated 302 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:04.622057 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6222406-b3be-4a70-bda7-8abf17c5e98d None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:04.623097 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5690f1-4d27-4eb1-a3c4-d35f5cdea65b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:04.624349 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5690f1-4d27-4eb1-a3c4-d35f5cdea65b None admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:04.624827 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5690f1-4d27-4eb1-a3c4-d35f5cdea65b None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:04.625298 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5690f1-4d27-4eb1-a3c4-d35f5cdea65b None admin] PATH_INFO: `/v3/projects/7d6be68cf8b046d5b03fba3c5a367c11/groups/28f5845ddc6847b38fd62490f9dce3d9/roles/ed6466d17fb7403c8491bb7f1ef95fc1` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:04.627796 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5690f1-4d27-4eb1-a3c4-d35f5cdea65b None admin] RBAC: Authorizing `identity:create_grant(project_id=7d6be68cf8b046d5b03fba3c5a367c11, group_id=28f5845ddc6847b38fd62490f9dce3d9, role_id=ed6466d17fb7403c8491bb7f1ef95fc1)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:04.629445 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5690f1-4d27-4eb1-a3c4-d35f5cdea65b None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:04.682782 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:04.706630 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 206/406] 10.222.0.16 () {58 vars in 1442 bytes} [Wed Aug 24 09:08:04 2022] PUT /identity/v3/projects/7d6be68cf8b046d5b03fba3c5a367c11/groups/28f5845ddc6847b38fd62490f9dce3d9/roles/ed6466d17fb7403c8491bb7f1ef95fc1 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:08:04.707548 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5690f1-4d27-4eb1-a3c4-d35f5cdea65b None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:04.724723 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6222406-b3be-4a70-bda7-8abf17c5e98d None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:04.731815 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:04.766902 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc29cd6-8f2d-44f9-8dfa-1a896cbf9c76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:04.767884 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc29cd6-8f2d-44f9-8dfa-1a896cbf9c76 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:04.768433 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc29cd6-8f2d-44f9-8dfa-1a896cbf9c76 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:04.768912 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc29cd6-8f2d-44f9-8dfa-1a896cbf9c76 None admin] PATH_INFO: `/v3/groups/28f5845ddc6847b38fd62490f9dce3d9` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:04.770540 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc29cd6-8f2d-44f9-8dfa-1a896cbf9c76 None admin] RBAC: Authorizing `identity:get_group(group_id=28f5845ddc6847b38fd62490f9dce3d9)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:04.772447 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc29cd6-8f2d-44f9-8dfa-1a896cbf9c76 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:04.774906 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 201/407] 10.222.0.16 () {56 vars in 1180 bytes} [Wed Aug 24 09:08:04 2022] GET /identity/v3/groups/28f5845ddc6847b38fd62490f9dce3d9 => generated 226 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:04.776063 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc29cd6-8f2d-44f9-8dfa-1a896cbf9c76 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:04.849729 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5690f1-4d27-4eb1-a3c4-d35f5cdea65b None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:04.887119 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c69bf2-aaa4-4d23-badd-1554b1bf8e3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:04.887795 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c69bf2-aaa4-4d23-badd-1554b1bf8e3c None admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:04.888078 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c69bf2-aaa4-4d23-badd-1554b1bf8e3c None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:04.888626 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c69bf2-aaa4-4d23-badd-1554b1bf8e3c None admin] PATH_INFO: `/v3/projects/99f7f068dc714797ac5d88928535f7a2/groups/28f5845ddc6847b38fd62490f9dce3d9/roles/136cdf78e00a426997dd4e37faff489b` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:04.890356 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c69bf2-aaa4-4d23-badd-1554b1bf8e3c None admin] RBAC: Authorizing `identity:create_grant(project_id=99f7f068dc714797ac5d88928535f7a2, group_id=28f5845ddc6847b38fd62490f9dce3d9, role_id=136cdf78e00a426997dd4e37faff489b)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:04.891748 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c69bf2-aaa4-4d23-badd-1554b1bf8e3c None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:04.892153 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc29cd6-8f2d-44f9-8dfa-1a896cbf9c76 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:04.924355 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:04.930031 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131f7309-c225-4c7a-b33c-adf963401f55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:04.930670 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131f7309-c225-4c7a-b33c-adf963401f55 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:04.930949 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131f7309-c225-4c7a-b33c-adf963401f55 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:04.931222 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131f7309-c225-4c7a-b33c-adf963401f55 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:04.941943 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131f7309-c225-4c7a-b33c-adf963401f55 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:08:04.946924 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-131f7309-c225-4c7a-b33c-adf963401f55 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:08:04.946924 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:08:04.946924 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:08:04.946924 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:08:04.946924 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:08:04.946924 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:08:04.946924 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:08:04.946924 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:08:04.946924 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:08:04.946924 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:08:04.946924 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:08:04.946924 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:08:04.946924 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:08:04.946924 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:08:04.946924 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:08:04.946924 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:08:04.948111 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:08:04.948111 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:08:04.948111 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:08:04.948111 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:08:04.948111 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:08:04.948111 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:08:04.948111 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:08:04.948111 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:08:04.948111 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:08:04.948111 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:08:04.948111 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:08:04.948111 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Aug 24 09:08:04.948111 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:08:04.948111 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:08:04.948111 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:08:04.948111 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:08:04.948945 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:08:04.948945 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:08:04.948945 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:08:04.948945 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:08:04.948945 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:08:04.948945 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:08:04.948945 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:08:04.948945 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:08:04.948945 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:08:04.948945 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:08:04.948945 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:08:04.948945 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Aug 24 09:08:04.948945 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 202/408] 10.222.0.16 () {56 vars in 1096 bytes} [Wed Aug 24 09:08:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:08:04.949709 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131f7309-c225-4c7a-b33c-adf963401f55 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:05.004962 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 207/409] 10.222.0.16 () {58 vars in 1442 bytes} [Wed Aug 24 09:08:04 2022] PUT /identity/v3/projects/99f7f068dc714797ac5d88928535f7a2/groups/28f5845ddc6847b38fd62490f9dce3d9/roles/136cdf78e00a426997dd4e37faff489b => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:08:05.005812 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c69bf2-aaa4-4d23-badd-1554b1bf8e3c None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:05.128080 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:05.138209 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131f7309-c225-4c7a-b33c-adf963401f55 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:05.161922 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c69bf2-aaa4-4d23-badd-1554b1bf8e3c None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:05.208802 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587155c9-7ec2-4410-a626-02b5c704b43a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:05.209448 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587155c9-7ec2-4410-a626-02b5c704b43a None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:05.209728 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587155c9-7ec2-4410-a626-02b5c704b43a None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:05.210048 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587155c9-7ec2-4410-a626-02b5c704b43a None admin] PATH_INFO: `/v3/projects/99f7f068dc714797ac5d88928535f7a2` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:05.211241 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587155c9-7ec2-4410-a626-02b5c704b43a None admin] RBAC: Authorizing `identity:get_project(project_id=99f7f068dc714797ac5d88928535f7a2)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:05.212832 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587155c9-7ec2-4410-a626-02b5c704b43a None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:05.214934 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 203/410] 10.222.0.16 () {56 vars in 1186 bytes} [Wed Aug 24 09:08:04 2022] GET /identity/v3/projects/99f7f068dc714797ac5d88928535f7a2 => generated 302 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:05.220913 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e5d208-1e3c-4356-8327-1f2bf92d7a27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:05.221656 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e5d208-1e3c-4356-8327-1f2bf92d7a27 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:05.221937 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e5d208-1e3c-4356-8327-1f2bf92d7a27 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:05.222212 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e5d208-1e3c-4356-8327-1f2bf92d7a27 None admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:05.223242 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e5d208-1e3c-4356-8327-1f2bf92d7a27 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:05.223552 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e5d208-1e3c-4356-8327-1f2bf92d7a27 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:05.223831 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587155c9-7ec2-4410-a626-02b5c704b43a None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:05.225102 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e5d208-1e3c-4356-8327-1f2bf92d7a27 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:05.234448 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 208/411] 10.222.0.16 () {56 vars in 1099 bytes} [Wed Aug 24 09:08:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:05.242268 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e5d208-1e3c-4356-8327-1f2bf92d7a27 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:05.294061 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:05.354651 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e5d208-1e3c-4356-8327-1f2bf92d7a27 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:05.395109 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587155c9-7ec2-4410-a626-02b5c704b43a None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:05.398804 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b957c45b-7b3a-447d-bf50-105440554a8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:05.399448 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b957c45b-7b3a-447d-bf50-105440554a8e None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:05.399756 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b957c45b-7b3a-447d-bf50-105440554a8e None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:05.400031 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b957c45b-7b3a-447d-bf50-105440554a8e None admin] PATH_INFO: `/v3/groups/9876c820bdd643c2bae408d77ffba313` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:05.402166 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b957c45b-7b3a-447d-bf50-105440554a8e None admin] RBAC: Authorizing `identity:get_group(group_id=9876c820bdd643c2bae408d77ffba313)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:05.404763 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b957c45b-7b3a-447d-bf50-105440554a8e None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:05.411767 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 209/412] 10.222.0.16 () {56 vars in 1180 bytes} [Wed Aug 24 09:08:05 2022] GET /identity/v3/groups/9876c820bdd643c2bae408d77ffba313 => generated 229 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:05.413343 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a492047a-f691-4aa3-8b4e-704b89bf5bca None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:05.413830 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a492047a-f691-4aa3-8b4e-704b89bf5bca None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:05.414288 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a492047a-f691-4aa3-8b4e-704b89bf5bca None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:05.419942 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 210/413] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:08:05 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:08:05.420725 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a492047a-f691-4aa3-8b4e-704b89bf5bca None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:05.424598 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:05.444389 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb63ff07-dad9-4952-aa7c-1a65ec738529 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:05.445400 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb63ff07-dad9-4952-aa7c-1a65ec738529 None admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:05.445880 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb63ff07-dad9-4952-aa7c-1a65ec738529 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:05.446352 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb63ff07-dad9-4952-aa7c-1a65ec738529 None admin] PATH_INFO: `/v3/projects/99f7f068dc714797ac5d88928535f7a2/groups/28f5845ddc6847b38fd62490f9dce3d9/roles/ed6466d17fb7403c8491bb7f1ef95fc1` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:05.448767 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb63ff07-dad9-4952-aa7c-1a65ec738529 None admin] RBAC: Authorizing `identity:create_grant(project_id=99f7f068dc714797ac5d88928535f7a2, group_id=28f5845ddc6847b38fd62490f9dce3d9, role_id=ed6466d17fb7403c8491bb7f1ef95fc1)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:05.450459 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb63ff07-dad9-4952-aa7c-1a65ec738529 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:05.529722 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a492047a-f691-4aa3-8b4e-704b89bf5bca None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:05.562055 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 204/414] 10.222.0.16 () {58 vars in 1442 bytes} [Wed Aug 24 09:08:05 2022] PUT /identity/v3/projects/99f7f068dc714797ac5d88928535f7a2/groups/28f5845ddc6847b38fd62490f9dce3d9/roles/ed6466d17fb7403c8491bb7f1ef95fc1 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 09:08:05.567019 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c96dc1d-190f-4ff5-8602-b3050d93de79 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:05.567019 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c96dc1d-190f-4ff5-8602-b3050d93de79 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:05.567019 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c96dc1d-190f-4ff5-8602-b3050d93de79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:05.587044 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b179e4b4-2f81-40c0-be05-eb0c8b0dd035 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:05.587798 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b179e4b4-2f81-40c0-be05-eb0c8b0dd035 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:05.588193 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b179e4b4-2f81-40c0-be05-eb0c8b0dd035 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:05.588519 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b179e4b4-2f81-40c0-be05-eb0c8b0dd035 None admin] PATH_INFO: `/v3/projects/e359feabfca34b0e8852e124e6b4b5a1` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:05.589919 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b179e4b4-2f81-40c0-be05-eb0c8b0dd035 None admin] RBAC: Authorizing `identity:get_project(project_id=e359feabfca34b0e8852e124e6b4b5a1)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:05.593654 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b179e4b4-2f81-40c0-be05-eb0c8b0dd035 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:05.597164 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 211/415] 10.222.0.16 () {56 vars in 1186 bytes} [Wed Aug 24 09:08:05 2022] GET /identity/v3/projects/e359feabfca34b0e8852e124e6b4b5a1 => generated 344 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:05.598158 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-1c96dc1d-190f-4ff5-8602-b3050d93de79 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:08:05.599501 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-1c96dc1d-190f-4ff5-8602-b3050d93de79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:05.603095 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b179e4b4-2f81-40c0-be05-eb0c8b0dd035 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:05.690659 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 205/416] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:08:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:08:05.731489 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:05.767654 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b179e4b4-2f81-40c0-be05-eb0c8b0dd035 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:05.804577 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3016519-e61e-4a99-ae50-3c529452e93a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:05.805610 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3016519-e61e-4a99-ae50-3c529452e93a None admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:05.806091 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3016519-e61e-4a99-ae50-3c529452e93a None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:05.806582 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3016519-e61e-4a99-ae50-3c529452e93a None admin] PATH_INFO: `/v3/projects/e359feabfca34b0e8852e124e6b4b5a1/groups/9876c820bdd643c2bae408d77ffba313/roles/15f634d4a2d247ddab90c458e5e1c141` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:05.808714 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3016519-e61e-4a99-ae50-3c529452e93a None admin] RBAC: Authorizing `identity:create_grant(project_id=e359feabfca34b0e8852e124e6b4b5a1, group_id=9876c820bdd643c2bae408d77ffba313, role_id=15f634d4a2d247ddab90c458e5e1c141)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:05.810430 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3016519-e61e-4a99-ae50-3c529452e93a None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:05.895488 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 212/417] 10.222.0.16 () {58 vars in 1442 bytes} [Wed Aug 24 09:08:05 2022] PUT /identity/v3/projects/e359feabfca34b0e8852e124e6b4b5a1/groups/9876c820bdd643c2bae408d77ffba313/roles/15f634d4a2d247ddab90c458e5e1c141 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 09:08:05.990102 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c96dc1d-190f-4ff5-8602-b3050d93de79 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:06.052925 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:06.109208 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c96dc1d-190f-4ff5-8602-b3050d93de79 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:06.149883 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecbb1cf-aeb0-459d-ac44-5f0835f7d3d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:06.150524 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecbb1cf-aeb0-459d-ac44-5f0835f7d3d1 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:06.150809 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecbb1cf-aeb0-459d-ac44-5f0835f7d3d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:06.151085 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecbb1cf-aeb0-459d-ac44-5f0835f7d3d1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:06.162684 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecbb1cf-aeb0-459d-ac44-5f0835f7d3d1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:08:06.167548 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-6ecbb1cf-aeb0-459d-ac44-5f0835f7d3d1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 09:08:06.167548 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:08:06.167548 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:08:06.167548 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:08:06.167548 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:08:06.167548 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:08:06.167548 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:08:06.167548 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:08:06.167548 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:08:06.167548 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:08:06.167548 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:08:06.167548 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:08:06.167548 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:08:06.167548 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:08:06.167548 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:08:06.167548 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:08:06.169215 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:08:06.169215 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:08:06.169215 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:08:06.169215 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:08:06.169215 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:08:06.169215 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:08:06.169215 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:08:06.169215 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:08:06.169215 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:08:06.169215 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:08:06.169215 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:08:06.169215 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Aug 24 09:08:06.169215 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:08:06.169215 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:08:06.169215 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:08:06.169215 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:08:06.170044 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:08:06.170044 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:08:06.170044 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:08:06.170044 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:08:06.170044 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:08:06.170044 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:08:06.170044 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:08:06.170044 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:08:06.170044 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:08:06.170044 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:08:06.170044 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 09:08:06.170044 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Aug 24 09:08:06.170044 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 206/418] 10.222.0.16 () {56 vars in 1099 bytes} [Wed Aug 24 09:08:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:08:06.177371 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3016519-e61e-4a99-ae50-3c529452e93a None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:06.300646 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3016519-e61e-4a99-ae50-3c529452e93a None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:06.353398 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6b5e9d-eb52-4a57-a8db-f73b9e61175a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:06.354088 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6b5e9d-eb52-4a57-a8db-f73b9e61175a None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:06.354376 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6b5e9d-eb52-4a57-a8db-f73b9e61175a None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:06.354650 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6b5e9d-eb52-4a57-a8db-f73b9e61175a None admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:06.355322 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6b5e9d-eb52-4a57-a8db-f73b9e61175a None admin] RBAC: Adding query filter params (name=member) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:06.355704 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6b5e9d-eb52-4a57-a8db-f73b9e61175a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:06.357590 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6b5e9d-eb52-4a57-a8db-f73b9e61175a None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:06.359889 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:06.377807 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 213/419] 10.222.0.16 () {56 vars in 1101 bytes} [Wed Aug 24 09:08:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:06.383333 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecbb1cf-aeb0-459d-ac44-5f0835f7d3d1 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:06.551527 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecbb1cf-aeb0-459d-ac44-5f0835f7d3d1 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:06.560587 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:06.585934 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2bd888-69cf-4388-b5e1-b1d51933743a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:06.586559 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2bd888-69cf-4388-b5e1-b1d51933743a None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:06.586842 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2bd888-69cf-4388-b5e1-b1d51933743a None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:06.587182 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2bd888-69cf-4388-b5e1-b1d51933743a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:06.668599 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2bd888-69cf-4388-b5e1-b1d51933743a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:06.670121 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2bd888-69cf-4388-b5e1-b1d51933743a None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:06.791679 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 207/420] 10.222.0.16 () {58 vars in 1282 bytes} [Wed Aug 24 09:08:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 09:08:06.797859 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6b5e9d-eb52-4a57-a8db-f73b9e61175a None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:06.951510 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6b5e9d-eb52-4a57-a8db-f73b9e61175a None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:06.996219 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c4581b-7541-456f-a69f-0e7b44c4b228 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:06.997405 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c4581b-7541-456f-a69f-0e7b44c4b228 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:06.997687 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c4581b-7541-456f-a69f-0e7b44c4b228 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:06.997963 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c4581b-7541-456f-a69f-0e7b44c4b228 None admin] PATH_INFO: `/v3/projects/7d6be68cf8b046d5b03fba3c5a367c11` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:07.004285 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c4581b-7541-456f-a69f-0e7b44c4b228 None admin] RBAC: Authorizing `identity:get_project(project_id=7d6be68cf8b046d5b03fba3c5a367c11)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:07.005754 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c4581b-7541-456f-a69f-0e7b44c4b228 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:07.007766 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 214/421] 10.222.0.16 () {56 vars in 1186 bytes} [Wed Aug 24 09:08:06 2022] GET /identity/v3/projects/7d6be68cf8b046d5b03fba3c5a367c11 => generated 298 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:07.013402 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2bd888-69cf-4388-b5e1-b1d51933743a None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:07.034129 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:07.056594 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b64754a-ae10-492a-a03e-6ad24a83e0e0 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:07.056921 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b64754a-ae10-492a-a03e-6ad24a83e0e0 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:07.057148 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b64754a-ae10-492a-a03e-6ad24a83e0e0 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:07.059745 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 215/422] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:08:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:08:07.067659 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babe4adc-35b1-49b4-b9e6-660a4a3264a7 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:07.068001 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babe4adc-35b1-49b4-b9e6-660a4a3264a7 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:07.068186 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babe4adc-35b1-49b4-b9e6-660a4a3264a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:07.102326 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-babe4adc-35b1-49b4-b9e6-660a4a3264a7 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:08:07.103801 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-babe4adc-35b1-49b4-b9e6-660a4a3264a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:07.151733 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2bd888-69cf-4388-b5e1-b1d51933743a None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:07.192398 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7d2921-b99d-46f6-b44e-561da6bd1630 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:07.194132 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7d2921-b99d-46f6-b44e-561da6bd1630 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:07.194825 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7d2921-b99d-46f6-b44e-561da6bd1630 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:07.195945 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7d2921-b99d-46f6-b44e-561da6bd1630 None admin] PATH_INFO: `/v3/groups/28f5845ddc6847b38fd62490f9dce3d9` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:07.199439 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7d2921-b99d-46f6-b44e-561da6bd1630 None admin] RBAC: Authorizing `identity:get_group(group_id=28f5845ddc6847b38fd62490f9dce3d9)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:07.201162 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7d2921-b99d-46f6-b44e-561da6bd1630 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:07.205259 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 208/423] 10.222.0.16 () {56 vars in 1180 bytes} [Wed Aug 24 09:08:07 2022] GET /identity/v3/groups/28f5845ddc6847b38fd62490f9dce3d9 => generated 226 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:07.211817 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:07.216712 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7d2921-b99d-46f6-b44e-561da6bd1630 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:07.226099 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 216/424] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:08:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:08:07.325865 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:07.362789 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7d2921-b99d-46f6-b44e-561da6bd1630 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:07.411550 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb47f4e-e079-4939-aa62-328b75a17d4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:07.412184 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb47f4e-e079-4939-aa62-328b75a17d4f None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:07.412466 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb47f4e-e079-4939-aa62-328b75a17d4f None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:07.412740 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb47f4e-e079-4939-aa62-328b75a17d4f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:07.413467 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb47f4e-e079-4939-aa62-328b75a17d4f None admin] RBAC: Adding query filter params (group.id=28f5845ddc6847b38fd62490f9dce3d9, role.id=136cdf78e00a426997dd4e37faff489b, scope.project.id=7d6be68cf8b046d5b03fba3c5a367c11) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:07.413773 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb47f4e-e079-4939-aa62-328b75a17d4f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:07.415203 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb47f4e-e079-4939-aa62-328b75a17d4f None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:07.424330 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 209/425] 10.222.0.16 () {56 vars in 1376 bytes} [Wed Aug 24 09:08:07 2022] GET /identity/v3/role_assignments?group.id=28f5845ddc6847b38fd62490f9dce3d9&scope.project.id=7d6be68cf8b046d5b03fba3c5a367c11&role.id=136cdf78e00a426997dd4e37faff489b => generated 611 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:07.480369 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faea8784-11fe-46cf-8009-6417d3f458c6 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:07.481466 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faea8784-11fe-46cf-8009-6417d3f458c6 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:07.481936 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faea8784-11fe-46cf-8009-6417d3f458c6 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:07.483827 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 217/426] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:08:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:08:07.489795 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4eb45a-8e81-4852-90ac-6b1b103b0bea None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:07.490082 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4eb45a-8e81-4852-90ac-6b1b103b0bea None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:07.490358 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4eb45a-8e81-4852-90ac-6b1b103b0bea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:07.514612 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-0e4eb45a-8e81-4852-90ac-6b1b103b0bea None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:08:07.515841 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-0e4eb45a-8e81-4852-90ac-6b1b103b0bea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:07.576475 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faea8784-11fe-46cf-8009-6417d3f458c6 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:07.610045 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 210/427] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:08:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:08:07.757673 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faea8784-11fe-46cf-8009-6417d3f458c6 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:07.803748 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f68eb05-359b-4406-ad72-1a624f82abcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:07.804489 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f68eb05-359b-4406-ad72-1a624f82abcd None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:07.804687 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f68eb05-359b-4406-ad72-1a624f82abcd None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:07.804937 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f68eb05-359b-4406-ad72-1a624f82abcd None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:07.820053 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f68eb05-359b-4406-ad72-1a624f82abcd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:08:07.824891 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-9f68eb05-359b-4406-ad72-1a624f82abcd None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 09:08:07.824891 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:08:07.824891 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:08:07.824891 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:08:07.824891 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:08:07.824891 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:08:07.824891 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:08:07.824891 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:08:07.824891 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:08:07.824891 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:08:07.824891 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:08:07.824891 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:08:07.824891 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:08:07.824891 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:08:07.824891 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:08:07.824891 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:08:07.825847 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:08:07.825847 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:08:07.825847 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:08:07.825847 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:08:07.825847 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:08:07.825847 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:08:07.825847 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:08:07.825847 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:08:07.825847 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:08:07.825847 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:08:07.825847 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:08:07.825847 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 24 09:08:07.825847 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:08:07.825847 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:08:07.825847 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:08:07.825847 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:08:07.826684 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:08:07.826684 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:08:07.826684 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:08:07.826684 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:08:07.826684 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:08:07.826684 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:08:07.826684 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:08:07.826684 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:08:07.826684 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:08:07.826684 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:08:07.826684 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 09:08:07.826684 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 24 09:08:07.826684 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 218/428] 10.222.0.16 () {56 vars in 1114 bytes} [Wed Aug 24 09:08:07 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:08:07.832144 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4eb45a-8e81-4852-90ac-6b1b103b0bea None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:07.918407 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:07.932585 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f68eb05-359b-4406-ad72-1a624f82abcd None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:07.951772 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4eb45a-8e81-4852-90ac-6b1b103b0bea None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:07.995236 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:07.995649 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdf9e84-4de8-4f7b-bce2-569a6883fdbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:07.997140 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdf9e84-4de8-4f7b-bce2-569a6883fdbf None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:07.997140 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdf9e84-4de8-4f7b-bce2-569a6883fdbf None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:07.997414 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdf9e84-4de8-4f7b-bce2-569a6883fdbf None admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:07.998080 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdf9e84-4de8-4f7b-bce2-569a6883fdbf None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:07.998383 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdf9e84-4de8-4f7b-bce2-569a6883fdbf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:07.999783 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdf9e84-4de8-4f7b-bce2-569a6883fdbf None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:08.013405 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 211/429] 10.222.0.16 () {56 vars in 1111 bytes} [Wed Aug 24 09:08:07 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:08.016278 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdf9e84-4de8-4f7b-bce2-569a6883fdbf None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:08.048655 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f68eb05-359b-4406-ad72-1a624f82abcd None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:08.088613 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84508ef1-28a6-4274-a838-145273980d58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:08.090245 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84508ef1-28a6-4274-a838-145273980d58 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:08.091327 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84508ef1-28a6-4274-a838-145273980d58 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:08.092840 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84508ef1-28a6-4274-a838-145273980d58 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:08.105135 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84508ef1-28a6-4274-a838-145273980d58 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:08:08.110787 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-84508ef1-28a6-4274-a838-145273980d58 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 09:08:08.110787 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:08:08.110787 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:08:08.110787 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:08:08.110787 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:08:08.110787 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:08:08.110787 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:08:08.110787 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:08:08.110787 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:08:08.110787 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:08:08.110787 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:08:08.110787 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:08:08.110787 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:08:08.110787 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:08:08.110787 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:08:08.110787 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:08:08.111961 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:08:08.111961 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:08:08.111961 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:08:08.111961 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:08:08.111961 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:08:08.111961 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:08:08.111961 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:08:08.111961 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:08:08.111961 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:08:08.111961 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:08:08.111961 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:08:08.111961 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 24 09:08:08.111961 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:08:08.111961 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:08:08.111961 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:08:08.111961 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:08:08.112769 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:08:08.112769 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:08:08.112769 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:08:08.112769 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:08:08.112769 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:08:08.112769 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:08:08.112769 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:08:08.112769 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:08:08.112769 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:08:08.112769 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:08:08.112769 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 09:08:08.112769 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 24 09:08:08.114951 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 219/430] 10.222.0.16 () {56 vars in 1099 bytes} [Wed Aug 24 09:08:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:08:08.121011 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84508ef1-28a6-4274-a838-145273980d58 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:08.134775 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdf9e84-4de8-4f7b-bce2-569a6883fdbf None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:08.168734 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:08.176917 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fb76d0-7db2-40e2-98ad-e7d4eeea73d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:08.178170 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fb76d0-7db2-40e2-98ad-e7d4eeea73d9 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:08.178735 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fb76d0-7db2-40e2-98ad-e7d4eeea73d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:08.179027 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fb76d0-7db2-40e2-98ad-e7d4eeea73d9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:08.247806 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84508ef1-28a6-4274-a838-145273980d58 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:08.263208 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:08.271089 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fb76d0-7db2-40e2-98ad-e7d4eeea73d9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:08.272525 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fb76d0-7db2-40e2-98ad-e7d4eeea73d9 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:08.288273 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e234a4-b7c7-4c66-b492-899d547b931c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:08.288956 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e234a4-b7c7-4c66-b492-899d547b931c None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:08.289239 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e234a4-b7c7-4c66-b492-899d547b931c None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:08.289510 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e234a4-b7c7-4c66-b492-899d547b931c None admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:08.290230 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e234a4-b7c7-4c66-b492-899d547b931c None admin] RBAC: Adding query filter params (name=member) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:08.290519 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e234a4-b7c7-4c66-b492-899d547b931c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:08.291953 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e234a4-b7c7-4c66-b492-899d547b931c None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:08.301515 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 220/431] 10.222.0.16 () {56 vars in 1101 bytes} [Wed Aug 24 09:08:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:08:08.302741 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730286e3-38eb-4311-a0b3-5b143d2d138f None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:08.303025 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730286e3-38eb-4311-a0b3-5b143d2d138f None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:08.303299 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730286e3-38eb-4311-a0b3-5b143d2d138f None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:08.304827 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 221/432] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:08:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:08:08.305663 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cd429a-a3b1-45f1-a66a-d0d15d5b7f23 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:08.305976 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cd429a-a3b1-45f1-a66a-d0d15d5b7f23 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:08.306453 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cd429a-a3b1-45f1-a66a-d0d15d5b7f23 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:08.307769 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 222/433] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:08:08 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:08:08.308352 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cd429a-a3b1-45f1-a66a-d0d15d5b7f23 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:08.384514 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 212/434] 10.222.0.16 () {58 vars in 1282 bytes} [Wed Aug 24 09:08:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 09:08:08.384514 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e337a5-dea7-4658-8369-6fc9f546141e None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:08.384514 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e337a5-dea7-4658-8369-6fc9f546141e None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:08.384514 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e337a5-dea7-4658-8369-6fc9f546141e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:08.398762 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-17e337a5-dea7-4658-8369-6fc9f546141e None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:08:08.399975 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-17e337a5-dea7-4658-8369-6fc9f546141e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:08.416221 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cd429a-a3b1-45f1-a66a-d0d15d5b7f23 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:08.425063 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:08.449983 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efcd5ed-be80-419e-a148-64d00c15e157 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:08.450587 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efcd5ed-be80-419e-a148-64d00c15e157 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:08.450865 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efcd5ed-be80-419e-a148-64d00c15e157 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:08.451137 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efcd5ed-be80-419e-a148-64d00c15e157 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:08.466441 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 213/435] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:08:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:08:08.467771 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f908316-2ee1-4a6d-9477-cd10939129f1 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:08.468111 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f908316-2ee1-4a6d-9477-cd10939129f1 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:08.468385 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f908316-2ee1-4a6d-9477-cd10939129f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:08.485652 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-9f908316-2ee1-4a6d-9477-cd10939129f1 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:08:08.486839 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-9f908316-2ee1-4a6d-9477-cd10939129f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:08.497522 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:08.525649 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efcd5ed-be80-419e-a148-64d00c15e157 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:08.527207 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efcd5ed-be80-419e-a148-64d00c15e157 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:08.567626 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 214/436] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:08:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:08:08.567626 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f908316-2ee1-4a6d-9477-cd10939129f1 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:08.642470 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 223/437] 10.222.0.16 () {58 vars in 1282 bytes} [Wed Aug 24 09:08:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 09:08:08.648831 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efcd5ed-be80-419e-a148-64d00c15e157 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:08.675232 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:08.686915 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f908316-2ee1-4a6d-9477-cd10939129f1 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:08.731728 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:08.732765 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b74c83-c408-4331-aab1-931fadfcf79e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:08.733404 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b74c83-c408-4331-aab1-931fadfcf79e None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:08.733677 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b74c83-c408-4331-aab1-931fadfcf79e None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:08.733947 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b74c83-c408-4331-aab1-931fadfcf79e None admin] PATH_INFO: `/v3/projects/7d6be68cf8b046d5b03fba3c5a367c11` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:08.736699 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b74c83-c408-4331-aab1-931fadfcf79e None admin] RBAC: Authorizing `identity:get_project(project_id=7d6be68cf8b046d5b03fba3c5a367c11)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:08.738957 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b74c83-c408-4331-aab1-931fadfcf79e None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:08.741789 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 215/438] 10.222.0.16 () {56 vars in 1186 bytes} [Wed Aug 24 09:08:08 2022] GET /identity/v3/projects/7d6be68cf8b046d5b03fba3c5a367c11 => generated 298 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:08.744617 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b74c83-c408-4331-aab1-931fadfcf79e None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:08.764236 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efcd5ed-be80-419e-a148-64d00c15e157 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:08.799496 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad5ce2f-cfcf-471a-9466-8e3a3987c262 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:08.800164 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad5ce2f-cfcf-471a-9466-8e3a3987c262 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:08.800448 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad5ce2f-cfcf-471a-9466-8e3a3987c262 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:08.800722 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad5ce2f-cfcf-471a-9466-8e3a3987c262 None admin] PATH_INFO: `/v3/projects/99f7f068dc714797ac5d88928535f7a2` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:08.802417 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad5ce2f-cfcf-471a-9466-8e3a3987c262 None admin] RBAC: Authorizing `identity:get_project(project_id=99f7f068dc714797ac5d88928535f7a2)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:08.803916 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad5ce2f-cfcf-471a-9466-8e3a3987c262 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:08.805534 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 224/439] 10.222.0.16 () {56 vars in 1186 bytes} [Wed Aug 24 09:08:08 2022] GET /identity/v3/projects/99f7f068dc714797ac5d88928535f7a2 => generated 302 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:08.808879 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad5ce2f-cfcf-471a-9466-8e3a3987c262 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:08.859032 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b74c83-c408-4331-aab1-931fadfcf79e None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:08.893782 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d6e693-1a0d-4bdc-8315-c94944680caf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:08.894408 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d6e693-1a0d-4bdc-8315-c94944680caf None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:08.894685 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d6e693-1a0d-4bdc-8315-c94944680caf None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:08.894958 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d6e693-1a0d-4bdc-8315-c94944680caf None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:08.905772 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d6e693-1a0d-4bdc-8315-c94944680caf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:08:08.911125 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-e1d6e693-1a0d-4bdc-8315-c94944680caf None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 09:08:08.911125 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:08:08.911125 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:08:08.911125 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:08:08.911125 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:08:08.911125 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:08:08.911125 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:08:08.911125 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:08:08.911125 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:08:08.911125 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:08:08.911125 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:08:08.911125 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:08:08.911125 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:08:08.911125 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:08:08.911125 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:08:08.911125 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:08:08.912647 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:08:08.912647 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:08:08.912647 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:08:08.912647 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:08:08.912647 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:08:08.912647 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:08:08.912647 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:08:08.912647 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:08:08.912647 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:08:08.912647 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:08:08.912647 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:08:08.912647 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Aug 24 09:08:08.912647 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:08:08.912647 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:08:08.912647 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:08:08.912647 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:08:08.916533 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:08:08.916533 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:08:08.916533 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:08:08.916533 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:08:08.916533 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:08:08.916533 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:08:08.916533 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:08:08.916533 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:08:08.916533 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:08:08.916533 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:08:08.916533 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 09:08:08.916533 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Aug 24 09:08:08.916533 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 216/440] 10.222.0.16 () {56 vars in 1114 bytes} [Wed Aug 24 09:08:08 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 09:08:08.916533 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d6e693-1a0d-4bdc-8315-c94944680caf None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:08.925710 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad5ce2f-cfcf-471a-9466-8e3a3987c262 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:08.936391 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:08.959789 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36848561-9fa6-4e6c-8ce9-237f968aa47e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:08.960414 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36848561-9fa6-4e6c-8ce9-237f968aa47e None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:08.960688 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36848561-9fa6-4e6c-8ce9-237f968aa47e None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:08.960959 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36848561-9fa6-4e6c-8ce9-237f968aa47e None admin] PATH_INFO: `/v3/groups/28f5845ddc6847b38fd62490f9dce3d9` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:08.962057 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36848561-9fa6-4e6c-8ce9-237f968aa47e None admin] RBAC: Authorizing `identity:get_group(group_id=28f5845ddc6847b38fd62490f9dce3d9)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:08.963499 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36848561-9fa6-4e6c-8ce9-237f968aa47e None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:08.965111 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 225/441] 10.222.0.16 () {56 vars in 1180 bytes} [Wed Aug 24 09:08:08 2022] GET /identity/v3/groups/28f5845ddc6847b38fd62490f9dce3d9 => generated 226 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:08.965857 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36848561-9fa6-4e6c-8ce9-237f968aa47e None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:08.977977 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:09.013709 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d6e693-1a0d-4bdc-8315-c94944680caf None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:09.047027 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b956ba-cb33-41e3-844f-be88c7687593 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:09.047683 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b956ba-cb33-41e3-844f-be88c7687593 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:09.047965 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b956ba-cb33-41e3-844f-be88c7687593 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:09.048238 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b956ba-cb33-41e3-844f-be88c7687593 None admin] PATH_INFO: `/v3/groups/28f5845ddc6847b38fd62490f9dce3d9` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:09.049348 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b956ba-cb33-41e3-844f-be88c7687593 None admin] RBAC: Authorizing `identity:get_group(group_id=28f5845ddc6847b38fd62490f9dce3d9)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:09.050829 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b956ba-cb33-41e3-844f-be88c7687593 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:09.052363 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 217/442] 10.222.0.16 () {56 vars in 1180 bytes} [Wed Aug 24 09:08:08 2022] GET /identity/v3/groups/28f5845ddc6847b38fd62490f9dce3d9 => generated 226 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:09.053127 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b956ba-cb33-41e3-844f-be88c7687593 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:09.065037 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36848561-9fa6-4e6c-8ce9-237f968aa47e None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:09.098305 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf13e22-0c9c-4a7e-a1b8-380bade1a427 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:09.098931 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf13e22-0c9c-4a7e-a1b8-380bade1a427 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:09.099212 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf13e22-0c9c-4a7e-a1b8-380bade1a427 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:09.099488 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf13e22-0c9c-4a7e-a1b8-380bade1a427 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:09.110038 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf13e22-0c9c-4a7e-a1b8-380bade1a427 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:08:09.114763 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-daf13e22-0c9c-4a7e-a1b8-380bade1a427 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:08:09.114763 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:08:09.114763 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:08:09.114763 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:08:09.114763 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:08:09.114763 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:08:09.114763 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:08:09.114763 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:08:09.114763 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:08:09.114763 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:08:09.114763 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:08:09.114763 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:08:09.114763 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:08:09.114763 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:08:09.114763 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:08:09.114763 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:08:09.115737 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:08:09.115737 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:08:09.115737 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:08:09.115737 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:08:09.115737 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:08:09.115737 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:08:09.115737 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:08:09.115737 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:08:09.115737 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:08:09.115737 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:08:09.115737 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:08:09.115737 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 24 09:08:09.115737 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:08:09.115737 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:08:09.115737 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:08:09.115737 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:08:09.116573 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:08:09.116573 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:08:09.116573 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:08:09.116573 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:08:09.116573 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:08:09.116573 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:08:09.116573 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:08:09.116573 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:08:09.116573 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:08:09.116573 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:08:09.116573 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:08:09.116573 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 24 09:08:09.116573 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 226/443] 10.222.0.16 () {56 vars in 1096 bytes} [Wed Aug 24 09:08:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:08:09.117591 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf13e22-0c9c-4a7e-a1b8-380bade1a427 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:09.146564 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:09.158456 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b956ba-cb33-41e3-844f-be88c7687593 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:09.185966 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:09.193542 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145ae80c-6321-4c9a-917f-5362eeddd3bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:09.194159 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145ae80c-6321-4c9a-917f-5362eeddd3bc None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:09.194438 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145ae80c-6321-4c9a-917f-5362eeddd3bc None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:09.194711 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145ae80c-6321-4c9a-917f-5362eeddd3bc None admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:09.195378 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145ae80c-6321-4c9a-917f-5362eeddd3bc None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:09.195714 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145ae80c-6321-4c9a-917f-5362eeddd3bc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:09.197106 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145ae80c-6321-4c9a-917f-5362eeddd3bc None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:09.206176 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 218/444] 10.222.0.16 () {56 vars in 1111 bytes} [Wed Aug 24 09:08:09 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:09.206999 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145ae80c-6321-4c9a-917f-5362eeddd3bc None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:09.221852 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf13e22-0c9c-4a7e-a1b8-380bade1a427 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:09.256278 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1f931e-0f9c-463b-a855-a1c34957b247 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:09.256876 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1f931e-0f9c-463b-a855-a1c34957b247 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:09.257154 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1f931e-0f9c-463b-a855-a1c34957b247 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:09.257428 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1f931e-0f9c-463b-a855-a1c34957b247 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:09.258153 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1f931e-0f9c-463b-a855-a1c34957b247 None admin] RBAC: Adding query filter params (group.id=28f5845ddc6847b38fd62490f9dce3d9, role.id=ed6466d17fb7403c8491bb7f1ef95fc1, scope.project.id=7d6be68cf8b046d5b03fba3c5a367c11) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:09.258467 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1f931e-0f9c-463b-a855-a1c34957b247 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:09.259918 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1f931e-0f9c-463b-a855-a1c34957b247 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:09.265581 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 227/445] 10.222.0.16 () {56 vars in 1376 bytes} [Wed Aug 24 09:08:09 2022] GET /identity/v3/role_assignments?group.id=28f5845ddc6847b38fd62490f9dce3d9&scope.project.id=7d6be68cf8b046d5b03fba3c5a367c11&role.id=ed6466d17fb7403c8491bb7f1ef95fc1 => generated 611 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:09.266346 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1f931e-0f9c-463b-a855-a1c34957b247 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:09.315562 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145ae80c-6321-4c9a-917f-5362eeddd3bc None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:09.358730 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225fe1af-bd6c-4686-85d6-dd5f2fd8283b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:09.359497 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225fe1af-bd6c-4686-85d6-dd5f2fd8283b None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:09.359833 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225fe1af-bd6c-4686-85d6-dd5f2fd8283b None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:09.360172 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225fe1af-bd6c-4686-85d6-dd5f2fd8283b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:09.361267 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225fe1af-bd6c-4686-85d6-dd5f2fd8283b None admin] RBAC: Adding query filter params (group.id=28f5845ddc6847b38fd62490f9dce3d9, role.id=136cdf78e00a426997dd4e37faff489b, scope.project.id=99f7f068dc714797ac5d88928535f7a2) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:09.361391 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225fe1af-bd6c-4686-85d6-dd5f2fd8283b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:09.362865 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225fe1af-bd6c-4686-85d6-dd5f2fd8283b None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:09.369734 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 219/446] 10.222.0.16 () {56 vars in 1376 bytes} [Wed Aug 24 09:08:09 2022] GET /identity/v3/role_assignments?group.id=28f5845ddc6847b38fd62490f9dce3d9&scope.project.id=99f7f068dc714797ac5d88928535f7a2&role.id=136cdf78e00a426997dd4e37faff489b => generated 611 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:09.370774 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225fe1af-bd6c-4686-85d6-dd5f2fd8283b None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:09.396972 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:09.397990 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1f931e-0f9c-463b-a855-a1c34957b247 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:09.447358 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5cc04a-30b8-44a5-b0ba-a00eb5e74cb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:09.448292 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5cc04a-30b8-44a5-b0ba-a00eb5e74cb0 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:09.448745 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5cc04a-30b8-44a5-b0ba-a00eb5e74cb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:09.449187 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5cc04a-30b8-44a5-b0ba-a00eb5e74cb0 None admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:09.449916 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5cc04a-30b8-44a5-b0ba-a00eb5e74cb0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:09.450220 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5cc04a-30b8-44a5-b0ba-a00eb5e74cb0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:09.452847 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5cc04a-30b8-44a5-b0ba-a00eb5e74cb0 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:09.454825 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:09.466292 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 228/447] 10.222.0.16 () {56 vars in 1099 bytes} [Wed Aug 24 09:08:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:09.474118 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5cc04a-30b8-44a5-b0ba-a00eb5e74cb0 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:09.519265 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225fe1af-bd6c-4686-85d6-dd5f2fd8283b None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:09.563055 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70d3130-72f6-4436-934b-d235ec9d43d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:09.563607 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70d3130-72f6-4436-934b-d235ec9d43d3 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:09.563997 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70d3130-72f6-4436-934b-d235ec9d43d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:09.564269 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70d3130-72f6-4436-934b-d235ec9d43d3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:09.613199 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5cc04a-30b8-44a5-b0ba-a00eb5e74cb0 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:09.645741 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70d3130-72f6-4436-934b-d235ec9d43d3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:09.647199 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70d3130-72f6-4436-934b-d235ec9d43d3 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:09.652208 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a480be-f4fe-46ec-8938-dd268b2cf943 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:09.654494 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a480be-f4fe-46ec-8938-dd268b2cf943 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:09.654772 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a480be-f4fe-46ec-8938-dd268b2cf943 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:09.655042 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a480be-f4fe-46ec-8938-dd268b2cf943 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:09.663954 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:09.724900 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:09.734197 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a480be-f4fe-46ec-8938-dd268b2cf943 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:09.735959 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a480be-f4fe-46ec-8938-dd268b2cf943 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:09.763297 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 220/448] 10.222.0.16 () {58 vars in 1282 bytes} [Wed Aug 24 09:08:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 09:08:09.769004 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70d3130-72f6-4436-934b-d235ec9d43d3 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:09.858575 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 229/449] 10.222.0.16 () {58 vars in 1282 bytes} [Wed Aug 24 09:08:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 09:08:09.864530 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a480be-f4fe-46ec-8938-dd268b2cf943 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:09.878913 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70d3130-72f6-4436-934b-d235ec9d43d3 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:09.893028 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:09.912870 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d180d248-05d0-4809-87bf-91e90315dee1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:09.913492 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d180d248-05d0-4809-87bf-91e90315dee1 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:09.913773 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d180d248-05d0-4809-87bf-91e90315dee1 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:09.914047 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d180d248-05d0-4809-87bf-91e90315dee1 None admin] PATH_INFO: `/v3/projects/99f7f068dc714797ac5d88928535f7a2` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:09.915170 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d180d248-05d0-4809-87bf-91e90315dee1 None admin] RBAC: Authorizing `identity:get_project(project_id=99f7f068dc714797ac5d88928535f7a2)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:09.916638 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d180d248-05d0-4809-87bf-91e90315dee1 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:09.918774 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 221/450] 10.222.0.16 () {56 vars in 1186 bytes} [Wed Aug 24 09:08:09 2022] GET /identity/v3/projects/99f7f068dc714797ac5d88928535f7a2 => generated 302 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:08:09.924261 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d180d248-05d0-4809-87bf-91e90315dee1 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:09.956078 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:09.961786 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a480be-f4fe-46ec-8938-dd268b2cf943 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:09.994799 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ababa9-196e-4ecb-900a-37079a8d9e5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:09.995418 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ababa9-196e-4ecb-900a-37079a8d9e5f None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:09.995726 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ababa9-196e-4ecb-900a-37079a8d9e5f None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:09.995998 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ababa9-196e-4ecb-900a-37079a8d9e5f None admin] PATH_INFO: `/v3/projects/e359feabfca34b0e8852e124e6b4b5a1` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:09.997667 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ababa9-196e-4ecb-900a-37079a8d9e5f None admin] RBAC: Authorizing `identity:get_project(project_id=e359feabfca34b0e8852e124e6b4b5a1)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:09.999102 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ababa9-196e-4ecb-900a-37079a8d9e5f None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:10.000736 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 230/451] 10.222.0.16 () {56 vars in 1186 bytes} [Wed Aug 24 09:08:09 2022] GET /identity/v3/projects/e359feabfca34b0e8852e124e6b4b5a1 => generated 344 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:08:10.006120 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ababa9-196e-4ecb-900a-37079a8d9e5f None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:10.021855 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d180d248-05d0-4809-87bf-91e90315dee1 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:10.055573 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e336881d-c3c9-4c5f-9368-f917465d43e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:10.056220 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e336881d-c3c9-4c5f-9368-f917465d43e5 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:10.056496 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e336881d-c3c9-4c5f-9368-f917465d43e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:10.056765 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e336881d-c3c9-4c5f-9368-f917465d43e5 None admin] PATH_INFO: `/v3/groups/28f5845ddc6847b38fd62490f9dce3d9` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:10.057875 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e336881d-c3c9-4c5f-9368-f917465d43e5 None admin] RBAC: Authorizing `identity:get_group(group_id=28f5845ddc6847b38fd62490f9dce3d9)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:10.059319 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e336881d-c3c9-4c5f-9368-f917465d43e5 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:10.060892 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 222/452] 10.222.0.16 () {56 vars in 1180 bytes} [Wed Aug 24 09:08:09 2022] GET /identity/v3/groups/28f5845ddc6847b38fd62490f9dce3d9 => generated 226 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:08:10.066526 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e336881d-c3c9-4c5f-9368-f917465d43e5 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:10.103851 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ababa9-196e-4ecb-900a-37079a8d9e5f None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:10.115534 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:10.138204 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c0601a-9fe9-476b-af25-45ac15549547 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:10.138829 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c0601a-9fe9-476b-af25-45ac15549547 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:10.139107 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c0601a-9fe9-476b-af25-45ac15549547 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:10.139380 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c0601a-9fe9-476b-af25-45ac15549547 None admin] PATH_INFO: `/v3/groups/9876c820bdd643c2bae408d77ffba313` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:10.140536 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c0601a-9fe9-476b-af25-45ac15549547 None admin] RBAC: Authorizing `identity:get_group(group_id=9876c820bdd643c2bae408d77ffba313)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:10.141969 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c0601a-9fe9-476b-af25-45ac15549547 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:10.143542 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 231/453] 10.222.0.16 () {56 vars in 1180 bytes} [Wed Aug 24 09:08:10 2022] GET /identity/v3/groups/9876c820bdd643c2bae408d77ffba313 => generated 229 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:08:10.149154 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c0601a-9fe9-476b-af25-45ac15549547 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:10.165656 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e336881d-c3c9-4c5f-9368-f917465d43e5 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:10.179593 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:10.199363 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0986e0a-6e39-455d-af96-9d7ca6a689f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:10.199992 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0986e0a-6e39-455d-af96-9d7ca6a689f6 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:10.200276 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0986e0a-6e39-455d-af96-9d7ca6a689f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:10.200549 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0986e0a-6e39-455d-af96-9d7ca6a689f6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:10.201277 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0986e0a-6e39-455d-af96-9d7ca6a689f6 None admin] RBAC: Adding query filter params (group.id=28f5845ddc6847b38fd62490f9dce3d9, role.id=ed6466d17fb7403c8491bb7f1ef95fc1, scope.project.id=99f7f068dc714797ac5d88928535f7a2) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:10.201592 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0986e0a-6e39-455d-af96-9d7ca6a689f6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:10.203009 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0986e0a-6e39-455d-af96-9d7ca6a689f6 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:10.209367 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 223/454] 10.222.0.16 () {56 vars in 1376 bytes} [Wed Aug 24 09:08:10 2022] GET /identity/v3/role_assignments?group.id=28f5845ddc6847b38fd62490f9dce3d9&scope.project.id=99f7f068dc714797ac5d88928535f7a2&role.id=ed6466d17fb7403c8491bb7f1ef95fc1 => generated 611 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:08:10.251352 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c0601a-9fe9-476b-af25-45ac15549547 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:10.288004 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f3bb17-bb90-40ec-9e4b-767e1069ac23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:10.288620 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f3bb17-bb90-40ec-9e4b-767e1069ac23 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:10.288900 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f3bb17-bb90-40ec-9e4b-767e1069ac23 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:10.289175 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f3bb17-bb90-40ec-9e4b-767e1069ac23 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:10.289911 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f3bb17-bb90-40ec-9e4b-767e1069ac23 None admin] RBAC: Adding query filter params (group.id=9876c820bdd643c2bae408d77ffba313, role.id=15f634d4a2d247ddab90c458e5e1c141, scope.project.id=e359feabfca34b0e8852e124e6b4b5a1) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:10.290224 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f3bb17-bb90-40ec-9e4b-767e1069ac23 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:10.291692 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f3bb17-bb90-40ec-9e4b-767e1069ac23 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:10.307691 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 232/455] 10.222.0.16 () {56 vars in 1376 bytes} [Wed Aug 24 09:08:10 2022] GET /identity/v3/role_assignments?group.id=9876c820bdd643c2bae408d77ffba313&scope.project.id=e359feabfca34b0e8852e124e6b4b5a1&role.id=15f634d4a2d247ddab90c458e5e1c141 => generated 611 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:08:12.594031 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c42d74f-f1d0-45a1-9ba6-222e55ac3cb0 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:12.595031 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c42d74f-f1d0-45a1-9ba6-222e55ac3cb0 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:12.595521 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c42d74f-f1d0-45a1-9ba6-222e55ac3cb0 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:12.597298 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 224/456] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:08:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:08:12.605792 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2c4477-dec6-4be4-9e05-12812ddbdd76 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:12.606692 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2c4477-dec6-4be4-9e05-12812ddbdd76 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:12.607168 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2c4477-dec6-4be4-9e05-12812ddbdd76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:12.632654 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-be2c4477-dec6-4be4-9e05-12812ddbdd76 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:08:12.634199 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-be2c4477-dec6-4be4-9e05-12812ddbdd76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:12.737199 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 233/457] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:08:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 09:08:12.792233 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c1f776-9c0d-4e81-b9e4-821c743ee96c None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:12.793392 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c1f776-9c0d-4e81-b9e4-821c743ee96c None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:12.793895 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c1f776-9c0d-4e81-b9e4-821c743ee96c None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:12.795521 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 225/458] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:08:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:08:12.800742 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e745da7-197b-4061-bb6f-2333ccf46c96 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:12.801250 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e745da7-197b-4061-bb6f-2333ccf46c96 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:12.801734 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e745da7-197b-4061-bb6f-2333ccf46c96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:12.828612 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-7e745da7-197b-4061-bb6f-2333ccf46c96 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:08:12.830404 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-7e745da7-197b-4061-bb6f-2333ccf46c96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:12.857805 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:12.925194 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 234/459] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:08:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:08:13.045829 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c1f776-9c0d-4e81-b9e4-821c743ee96c None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:13.179432 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c1f776-9c0d-4e81-b9e4-821c743ee96c None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:13.205360 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:13.228003 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f337ed-a0e8-43c7-bb92-9238fb874cd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:13.228644 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f337ed-a0e8-43c7-bb92-9238fb874cd2 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:13.228929 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f337ed-a0e8-43c7-bb92-9238fb874cd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:13.229208 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f337ed-a0e8-43c7-bb92-9238fb874cd2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:13.247262 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f337ed-a0e8-43c7-bb92-9238fb874cd2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:13.248778 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f337ed-a0e8-43c7-bb92-9238fb874cd2 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:13.265012 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-a8f337ed-a0e8-43c7-bb92-9238fb874cd2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:08:13.265012 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:08:13.265012 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:08:13.265012 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:08:13.265012 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:08:13.265012 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:08:13.265012 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:08:13.265012 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:08:13.265012 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:08:13.265012 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:08:13.265012 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:08:13.265012 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:08:13.265012 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:08:13.265012 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:08:13.265012 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:08:13.265012 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:08:13.265960 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:08:13.265960 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:08:13.265960 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:08:13.265960 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:08:13.265960 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:08:13.265960 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:08:13.265960 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:08:13.265960 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Aug 24 09:08:13.265960 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:08:13.265960 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:08:13.265960 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:08:13.265960 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:08:13.265960 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:08:13.265960 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:08:13.265960 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:08:13.265960 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:08:13.265960 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:08:13.266807 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:08:13.266807 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:08:13.266807 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Aug 24 09:08:13.268907 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 226/460] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 24 09:08:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:08:13.274642 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e745da7-197b-4061-bb6f-2333ccf46c96 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:13.315165 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f337ed-a0e8-43c7-bb92-9238fb874cd2 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:13.442131 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e745da7-197b-4061-bb6f-2333ccf46c96 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:13.459168 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f337ed-a0e8-43c7-bb92-9238fb874cd2 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:13.482738 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bde4bc4-ac74-48d1-bc45-03ea329d2631 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:13.483779 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bde4bc4-ac74-48d1-bc45-03ea329d2631 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:13.484257 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bde4bc4-ac74-48d1-bc45-03ea329d2631 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:13.484723 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bde4bc4-ac74-48d1-bc45-03ea329d2631 None admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:13.485690 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bde4bc4-ac74-48d1-bc45-03ea329d2631 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:13.486305 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bde4bc4-ac74-48d1-bc45-03ea329d2631 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:13.487903 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bde4bc4-ac74-48d1-bc45-03ea329d2631 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:13.493559 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241d2713-ce09-4211-a7b0-d901a94699cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:13.494399 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241d2713-ce09-4211-a7b0-d901a94699cf None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:13.494887 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241d2713-ce09-4211-a7b0-d901a94699cf None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:13.495349 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241d2713-ce09-4211-a7b0-d901a94699cf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:13.506381 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 235/461] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Aug 24 09:08:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:13.507552 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf71412-2737-4529-8923-f57655218e39 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:13.507889 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf71412-2737-4529-8923-f57655218e39 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:13.508180 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf71412-2737-4529-8923-f57655218e39 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:13.509519 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 236/462] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:08:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:08:13.511830 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241d2713-ce09-4211-a7b0-d901a94699cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:13.512705 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7b90a9-7e82-48d0-a08c-95f80ea8d612 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:13.512983 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7b90a9-7e82-48d0-a08c-95f80ea8d612 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:13.513251 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7b90a9-7e82-48d0-a08c-95f80ea8d612 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:13.514139 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241d2713-ce09-4211-a7b0-d901a94699cf None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:13.514586 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 237/463] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:08:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:08:13.515638 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7b90a9-7e82-48d0-a08c-95f80ea8d612 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:13.529921 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-241d2713-ce09-4211-a7b0-d901a94699cf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:08:13.529921 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:08:13.529921 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:08:13.529921 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:08:13.529921 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:08:13.529921 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:08:13.529921 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:08:13.529921 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:08:13.529921 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:08:13.529921 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:08:13.529921 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:08:13.529921 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:08:13.529921 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:08:13.529921 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:08:13.529921 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:08:13.529921 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:08:13.530861 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:08:13.530861 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:08:13.530861 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:08:13.530861 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:08:13.530861 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:08:13.530861 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:08:13.530861 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:08:13.530861 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Aug 24 09:08:13.530861 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:08:13.530861 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:08:13.530861 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:08:13.530861 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:08:13.530861 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:08:13.530861 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:08:13.530861 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:08:13.530861 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:08:13.530861 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:08:13.532306 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:08:13.532306 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:08:13.532306 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Aug 24 09:08:13.532306 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 227/464] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 24 09:08:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:08:13.532306 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae82031-a383-46a0-b0be-9e0a58c214f4 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:13.532585 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae82031-a383-46a0-b0be-9e0a58c214f4 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:13.532650 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae82031-a383-46a0-b0be-9e0a58c214f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:13.541199 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:13.551481 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-eae82031-a383-46a0-b0be-9e0a58c214f4 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:08:13.552770 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-eae82031-a383-46a0-b0be-9e0a58c214f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:13.566776 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:13.625387 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7b90a9-7e82-48d0-a08c-95f80ea8d612 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:13.626038 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 228/465] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:08:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:08:13.634023 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a882d77-0843-49bb-96b4-36d1d5a8ad34 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:13.634023 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a882d77-0843-49bb-96b4-36d1d5a8ad34 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:13.634023 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a882d77-0843-49bb-96b4-36d1d5a8ad34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:13.652974 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-7a882d77-0843-49bb-96b4-36d1d5a8ad34 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:08:13.654154 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-7a882d77-0843-49bb-96b4-36d1d5a8ad34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:13.671330 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9436ab9d-ccd2-4011-99df-7164f89c3d7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:13.672116 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9436ab9d-ccd2-4011-99df-7164f89c3d7b None admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:13.672398 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9436ab9d-ccd2-4011-99df-7164f89c3d7b None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:13.672691 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9436ab9d-ccd2-4011-99df-7164f89c3d7b None admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:13.673480 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9436ab9d-ccd2-4011-99df-7164f89c3d7b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:13.674984 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9436ab9d-ccd2-4011-99df-7164f89c3d7b None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:13.729883 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 229/466] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:08:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:08:13.731264 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a882d77-0843-49bb-96b4-36d1d5a8ad34 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:13.797102 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 238/467] 10.222.0.16 () {60 vars in 1132 bytes} [Wed Aug 24 09:08:13 2022] POST /identity/v3/users => generated 246 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:08:13.825090 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:13.849169 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a882d77-0843-49bb-96b4-36d1d5a8ad34 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:13.900840 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2935d52d-cff4-4028-b790-46db91f8d86c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:13.901475 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2935d52d-cff4-4028-b790-46db91f8d86c None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:13.901753 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2935d52d-cff4-4028-b790-46db91f8d86c None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:13.902066 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2935d52d-cff4-4028-b790-46db91f8d86c None admin] PATH_INFO: `/v3/domains` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:13.903271 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2935d52d-cff4-4028-b790-46db91f8d86c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:13.903573 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2935d52d-cff4-4028-b790-46db91f8d86c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:13.905066 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2935d52d-cff4-4028-b790-46db91f8d86c None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:13.910567 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9436ab9d-ccd2-4011-99df-7164f89c3d7b None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:13.918761 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 230/468] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Aug 24 09:08:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:13.924902 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2935d52d-cff4-4028-b790-46db91f8d86c None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:14.015494 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:14.053982 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9436ab9d-ccd2-4011-99df-7164f89c3d7b None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:14.090802 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2935d52d-cff4-4028-b790-46db91f8d86c None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:14.096238 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0554d83c-6e0f-4e32-a4d1-7fb4548ce893 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:14.096886 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0554d83c-6e0f-4e32-a4d1-7fb4548ce893 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:14.097175 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0554d83c-6e0f-4e32-a4d1-7fb4548ce893 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:14.097453 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0554d83c-6e0f-4e32-a4d1-7fb4548ce893 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:14.111790 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0554d83c-6e0f-4e32-a4d1-7fb4548ce893 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:14.113377 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0554d83c-6e0f-4e32-a4d1-7fb4548ce893 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:14.130471 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ca4694-75d8-47ee-a275-1fc5851e74ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:14.131086 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ca4694-75d8-47ee-a275-1fc5851e74ba None admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:14.131364 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ca4694-75d8-47ee-a275-1fc5851e74ba None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:14.131680 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ca4694-75d8-47ee-a275-1fc5851e74ba None admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:14.131967 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-0554d83c-6e0f-4e32-a4d1-7fb4548ce893 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:08:14.131967 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:08:14.131967 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:08:14.131967 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:08:14.131967 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:08:14.131967 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:08:14.131967 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:08:14.131967 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:08:14.131967 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:08:14.131967 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:08:14.131967 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:08:14.131967 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:08:14.131967 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:08:14.131967 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:08:14.131967 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:08:14.131967 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:08:14.132824 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:08:14.132824 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:08:14.132824 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:08:14.132824 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:08:14.132824 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:08:14.132824 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:08:14.132824 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:08:14.132824 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 24 09:08:14.132824 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:08:14.132824 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:08:14.132824 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:08:14.132824 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:08:14.132824 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:08:14.132824 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:08:14.132824 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:08:14.132824 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:08:14.132824 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:08:14.133754 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:08:14.133754 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:08:14.133754 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 24 09:08:14.133928 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ca4694-75d8-47ee-a275-1fc5851e74ba None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:14.133997 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 239/469] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 24 09:08:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 09:08:14.136355 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ca4694-75d8-47ee-a275-1fc5851e74ba None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:14.139152 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0554d83c-6e0f-4e32-a4d1-7fb4548ce893 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:14.236811 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:14.248687 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0554d83c-6e0f-4e32-a4d1-7fb4548ce893 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:14.277836 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:14.293594 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52467eae-c07d-4f1e-bf38-81917af8f0ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:14.294627 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52467eae-c07d-4f1e-bf38-81917af8f0ed None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:14.295095 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52467eae-c07d-4f1e-bf38-81917af8f0ed None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:14.295549 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52467eae-c07d-4f1e-bf38-81917af8f0ed None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:14.308461 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52467eae-c07d-4f1e-bf38-81917af8f0ed None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:14.309936 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52467eae-c07d-4f1e-bf38-81917af8f0ed None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:14.327125 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-52467eae-c07d-4f1e-bf38-81917af8f0ed None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:08:14.327125 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:08:14.327125 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:08:14.327125 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:08:14.327125 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:08:14.327125 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:08:14.327125 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:08:14.327125 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:08:14.327125 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:08:14.327125 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:08:14.327125 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:08:14.327125 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:08:14.327125 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:08:14.327125 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:08:14.327125 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:08:14.327125 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:08:14.328135 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:08:14.328135 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:08:14.328135 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:08:14.328135 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:08:14.328135 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:08:14.328135 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:08:14.328135 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:08:14.328135 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 24 09:08:14.328135 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:08:14.328135 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:08:14.328135 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:08:14.328135 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:08:14.328135 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:08:14.328135 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:08:14.328135 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:08:14.328135 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:08:14.328135 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:08:14.329049 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:08:14.329049 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:08:14.329049 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 24 09:08:14.329049 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 240/470] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 24 09:08:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 09:08:14.332390 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52467eae-c07d-4f1e-bf38-81917af8f0ed None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:14.335101 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 231/471] 10.222.0.16 () {60 vars in 1132 bytes} [Wed Aug 24 09:08:13 2022] POST /identity/v3/users => generated 248 bytes in 411 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 09:08:14.337734 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d5c9f8-e1ed-4fbe-bf9d-603ea2e39646 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:14.338240 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d5c9f8-e1ed-4fbe-bf9d-603ea2e39646 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:14.338684 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d5c9f8-e1ed-4fbe-bf9d-603ea2e39646 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:14.341706 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 232/472] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:08:14 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:08:14.342495 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d5c9f8-e1ed-4fbe-bf9d-603ea2e39646 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:14.473393 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52467eae-c07d-4f1e-bf38-81917af8f0ed None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:14.475633 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d5c9f8-e1ed-4fbe-bf9d-603ea2e39646 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:14.520220 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e85f92-6c26-4a08-9c86-924518727e3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:14.521161 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e85f92-6c26-4a08-9c86-924518727e3c None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:14.521653 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e85f92-6c26-4a08-9c86-924518727e3c None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:14.522119 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e85f92-6c26-4a08-9c86-924518727e3c None admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:14.522943 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e85f92-6c26-4a08-9c86-924518727e3c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:14.523258 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e85f92-6c26-4a08-9c86-924518727e3c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:14.526912 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e85f92-6c26-4a08-9c86-924518727e3c None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:14.527053 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670b07ce-093f-4cfb-b4fb-81c562737f1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:14.527053 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670b07ce-093f-4cfb-b4fb-81c562737f1e None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:14.527053 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670b07ce-093f-4cfb-b4fb-81c562737f1e None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:14.527053 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670b07ce-093f-4cfb-b4fb-81c562737f1e None admin] PATH_INFO: `/v3/domains` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:14.528122 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670b07ce-093f-4cfb-b4fb-81c562737f1e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:14.528317 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670b07ce-093f-4cfb-b4fb-81c562737f1e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:14.529699 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670b07ce-093f-4cfb-b4fb-81c562737f1e None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:14.544457 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 233/474] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Aug 24 09:08:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:14.544636 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 241/474] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Aug 24 09:08:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:14.545535 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e722ec8-7501-40cb-a386-5204d5fbe50c None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:14.545832 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e722ec8-7501-40cb-a386-5204d5fbe50c None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:14.546088 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e722ec8-7501-40cb-a386-5204d5fbe50c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:14.549935 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e85f92-6c26-4a08-9c86-924518727e3c None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:14.566956 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-1e722ec8-7501-40cb-a386-5204d5fbe50c None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:08:14.568225 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-1e722ec8-7501-40cb-a386-5204d5fbe50c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:14.597001 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:14.637674 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:14.661508 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 234/475] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:08:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:08:14.663237 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e722ec8-7501-40cb-a386-5204d5fbe50c None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:14.672338 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e85f92-6c26-4a08-9c86-924518727e3c None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:14.713588 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15e507a-1547-4b90-ad55-b877f4593685 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:14.715212 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15e507a-1547-4b90-ad55-b877f4593685 None admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:14.715858 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15e507a-1547-4b90-ad55-b877f4593685 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:14.716336 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15e507a-1547-4b90-ad55-b877f4593685 None admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:14.717197 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15e507a-1547-4b90-ad55-b877f4593685 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:14.718923 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15e507a-1547-4b90-ad55-b877f4593685 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:14.785978 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e722ec8-7501-40cb-a386-5204d5fbe50c None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:14.822410 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3764455b-ed07-4edc-801b-9d67817c11d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:14.823012 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3764455b-ed07-4edc-801b-9d67817c11d7 None admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:14.823217 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3764455b-ed07-4edc-801b-9d67817c11d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:14.823495 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3764455b-ed07-4edc-801b-9d67817c11d7 None admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:14.824194 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3764455b-ed07-4edc-801b-9d67817c11d7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:14.828066 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3764455b-ed07-4edc-801b-9d67817c11d7 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:14.832559 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 242/476] 10.222.0.16 () {60 vars in 1132 bytes} [Wed Aug 24 09:08:14 2022] POST /identity/v3/users => generated 248 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:08:14.939493 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 235/477] 10.222.0.16 () {60 vars in 1132 bytes} [Wed Aug 24 09:08:14 2022] POST /identity/v3/users => generated 249 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:08:15.028849 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15e507a-1547-4b90-ad55-b877f4593685 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:15.135183 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:15.144683 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15e507a-1547-4b90-ad55-b877f4593685 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:15.178878 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b37f1a2-b2ce-4d74-94f4-d0903d6ce61b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:15.179953 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b37f1a2-b2ce-4d74-94f4-d0903d6ce61b None admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:15.180784 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b37f1a2-b2ce-4d74-94f4-d0903d6ce61b None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:15.181097 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b37f1a2-b2ce-4d74-94f4-d0903d6ce61b None admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:15.181728 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b37f1a2-b2ce-4d74-94f4-d0903d6ce61b None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:15.184002 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b37f1a2-b2ce-4d74-94f4-d0903d6ce61b None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:15.240060 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.sql.core [None req-8b37f1a2-b2ce-4d74-94f4-d0903d6ce61b None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 24 09:08:15.240060 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 24 09:08:15.240060 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [parameters: {'id': '24331a8c8e694aefb93249ee39023346', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 24 09:08:15.240060 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81602) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 24 09:08:15.257527 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-8b37f1a2-b2ce-4d74-94f4-d0903d6ce61b None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 24 09:08:15.257527 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:08:15.257527 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 09:08:15.257527 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 09:08:15.257527 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 09:08:15.257527 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 09:08:15.257527 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 09:08:15.257527 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 09:08:15.257527 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 09:08:15.257527 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application conn.query(q) Aug 24 09:08:15.257527 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 09:08:15.257527 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 09:08:15.257527 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 09:08:15.257527 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application result.read() Aug 24 09:08:15.257527 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 09:08:15.258425 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 09:08:15.258425 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 09:08:15.258425 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 09:08:15.258425 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 09:08:15.258425 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 09:08:15.258425 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 09:08:15.258425 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 09:08:15.258425 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 24 09:08:15.258425 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 24 09:08:15.258425 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 24 09:08:15.258425 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 24 09:08:15.258425 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:08:15.258425 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 24 09:08:15.258425 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 24 09:08:15.258425 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Aug 24 09:08:15.258425 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return ref.to_dict() Aug 24 09:08:15.258425 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 09:08:15.258425 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application next(self.gen) Aug 24 09:08:15.259369 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 24 09:08:15.259369 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application yield resource Aug 24 09:08:15.259369 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 09:08:15.259369 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application next(self.gen) Aug 24 09:08:15.259369 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 24 09:08:15.259369 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application self.session.rollback() Aug 24 09:08:15.259369 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 24 09:08:15.259369 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application self.force_reraise() Aug 24 09:08:15.259369 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 24 09:08:15.259369 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise self.value Aug 24 09:08:15.259369 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 24 09:08:15.259369 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 24 09:08:15.259369 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 24 09:08:15.259369 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application session.commit() Aug 24 09:08:15.259369 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 24 09:08:15.259369 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 24 09:08:15.259369 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 24 09:08:15.260396 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application self._prepare_impl() Aug 24 09:08:15.260396 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 24 09:08:15.260396 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application self.session.flush() Aug 24 09:08:15.260396 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 24 09:08:15.260396 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application self._flush(objects) Aug 24 09:08:15.260396 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 24 09:08:15.260396 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 24 09:08:15.260396 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 24 09:08:15.260396 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application compat.raise_( Aug 24 09:08:15.260396 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 09:08:15.260396 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception Aug 24 09:08:15.260396 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 24 09:08:15.260396 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application flush_context.execute() Aug 24 09:08:15.260396 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 24 09:08:15.260396 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rec.execute(self) Aug 24 09:08:15.260396 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 24 09:08:15.260396 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 24 09:08:15.261289 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 24 09:08:15.261289 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 24 09:08:15.261289 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 24 09:08:15.261289 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 24 09:08:15.261289 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 24 09:08:15.261289 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 24 09:08:15.261289 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 24 09:08:15.261289 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 24 09:08:15.261289 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 24 09:08:15.261289 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 24 09:08:15.261289 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 24 09:08:15.261289 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 24 09:08:15.261289 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 24 09:08:15.261289 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 24 09:08:15.261289 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 09:08:15.261289 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception Aug 24 09:08:15.262203 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 09:08:15.262203 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 09:08:15.262203 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 09:08:15.262203 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 09:08:15.262203 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 09:08:15.262203 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 09:08:15.262203 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 09:08:15.262203 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application conn.query(q) Aug 24 09:08:15.262203 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 09:08:15.262203 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 09:08:15.262203 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 09:08:15.262203 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application result.read() Aug 24 09:08:15.262203 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 09:08:15.262203 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 09:08:15.262203 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 09:08:15.262203 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 09:08:15.263042 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 09:08:15.263042 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 09:08:15.263042 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 09:08:15.263042 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 09:08:15.263042 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 24 09:08:15.263042 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 24 09:08:15.263042 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [parameters: {'id': '24331a8c8e694aefb93249ee39023346', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 24 09:08:15.263042 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 24 09:08:15.263042 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 24 09:08:15.263042 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 09:08:15.263042 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 24 09:08:15.263042 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:08:15.263042 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:08:15.263042 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:08:15.263042 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:08:15.263042 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:08:15.265714 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:08:15.265714 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:08:15.265714 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:08:15.265714 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:08:15.265714 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:08:15.265714 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:08:15.265714 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Aug 24 09:08:15.265714 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Aug 24 09:08:15.265714 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:08:15.265714 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:08:15.265714 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Aug 24 09:08:15.265714 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Aug 24 09:08:15.265714 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 24 09:08:15.265714 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 24 09:08:15.265714 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 24 09:08:15.265714 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 24 09:08:15.273152 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 243/478] 10.222.0.16 () {60 vars in 1131 bytes} [Wed Aug 24 09:08:15 2022] POST /identity/v3/roles => generated 144 bytes in 231 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:08:15.273296 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3764455b-ed07-4edc-801b-9d67817c11d7 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:15.318964 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:15.386873 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3764455b-ed07-4edc-801b-9d67817c11d7 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:15.425003 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d44f5b-700e-4e5f-b64f-ed9c508a448c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:15.425702 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d44f5b-700e-4e5f-b64f-ed9c508a448c None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:15.426032 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d44f5b-700e-4e5f-b64f-ed9c508a448c None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:15.426317 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d44f5b-700e-4e5f-b64f-ed9c508a448c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:15.437584 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d44f5b-700e-4e5f-b64f-ed9c508a448c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:08:15.442884 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-a3d44f5b-700e-4e5f-b64f-ed9c508a448c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 09:08:15.442884 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:08:15.442884 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:08:15.442884 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:08:15.442884 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:08:15.442884 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:08:15.442884 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:08:15.442884 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:08:15.442884 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:08:15.442884 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:08:15.442884 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:08:15.442884 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:08:15.442884 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:08:15.442884 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:08:15.442884 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:08:15.442884 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:08:15.444967 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:08:15.444967 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:08:15.444967 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:08:15.444967 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:08:15.444967 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:08:15.444967 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:08:15.444967 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:08:15.444967 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:08:15.444967 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:08:15.444967 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:08:15.444967 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:08:15.444967 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Aug 24 09:08:15.444967 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:08:15.444967 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:08:15.444967 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:08:15.444967 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:08:15.446834 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:08:15.446834 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:08:15.446834 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:08:15.446834 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:08:15.446834 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:08:15.446834 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:08:15.446834 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:08:15.446834 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:08:15.446834 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:08:15.446834 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:08:15.446834 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 09:08:15.446834 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Aug 24 09:08:15.448324 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 236/479] 10.222.0.16 () {56 vars in 1114 bytes} [Wed Aug 24 09:08:15 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:08:15.450636 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b37f1a2-b2ce-4d74-94f4-d0903d6ce61b None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:15.557651 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b37f1a2-b2ce-4d74-94f4-d0903d6ce61b None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:15.593814 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efd24ef-bb7b-404d-9abf-1191c13321be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:15.594472 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efd24ef-bb7b-404d-9abf-1191c13321be None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:15.594751 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efd24ef-bb7b-404d-9abf-1191c13321be None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:15.595027 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efd24ef-bb7b-404d-9abf-1191c13321be None admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:15.595793 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efd24ef-bb7b-404d-9abf-1191c13321be None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:15.596102 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efd24ef-bb7b-404d-9abf-1191c13321be None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:15.597537 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efd24ef-bb7b-404d-9abf-1191c13321be None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:15.611773 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 244/480] 10.222.0.16 () {56 vars in 1111 bytes} [Wed Aug 24 09:08:15 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:16.065760 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c57033-21db-4d95-b73c-1e610acfc4fa None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:16.066756 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c57033-21db-4d95-b73c-1e610acfc4fa None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:16.067241 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c57033-21db-4d95-b73c-1e610acfc4fa None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:16.069030 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 237/481] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:08:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:08:16.074681 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538e02ee-00d9-480b-b32f-393823520150 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:16.075509 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538e02ee-00d9-480b-b32f-393823520150 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:16.075999 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538e02ee-00d9-480b-b32f-393823520150 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:16.100652 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-538e02ee-00d9-480b-b32f-393823520150 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:08:16.102135 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-538e02ee-00d9-480b-b32f-393823520150 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:16.123854 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:16.193063 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 245/482] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:08:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:08:16.584952 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c57033-21db-4d95-b73c-1e610acfc4fa None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:16.706838 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:16.713322 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c57033-21db-4d95-b73c-1e610acfc4fa None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:16.750814 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7191fc72-5d2b-48ca-b065-44dd9b337116 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:16.751447 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7191fc72-5d2b-48ca-b065-44dd9b337116 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:16.751755 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7191fc72-5d2b-48ca-b065-44dd9b337116 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:16.752025 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7191fc72-5d2b-48ca-b065-44dd9b337116 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:16.766858 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7191fc72-5d2b-48ca-b065-44dd9b337116 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:08:16.771629 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-7191fc72-5d2b-48ca-b065-44dd9b337116 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 09:08:16.771629 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:08:16.771629 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:08:16.771629 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:08:16.771629 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:08:16.771629 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:08:16.771629 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:08:16.771629 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:08:16.771629 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:08:16.771629 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:08:16.771629 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:08:16.771629 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:08:16.771629 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:08:16.771629 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:08:16.771629 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:08:16.771629 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:08:16.772534 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:08:16.772534 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:08:16.772534 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:08:16.772534 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:08:16.772534 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:08:16.772534 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:08:16.772534 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:08:16.772534 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:08:16.772534 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:08:16.772534 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:08:16.772534 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:08:16.772534 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Aug 24 09:08:16.772534 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:08:16.772534 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:08:16.772534 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:08:16.772534 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:08:16.773359 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:08:16.773359 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:08:16.773359 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:08:16.773359 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:08:16.773359 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:08:16.773359 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:08:16.773359 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:08:16.773359 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:08:16.773359 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:08:16.773359 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:08:16.773359 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 09:08:16.773359 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Aug 24 09:08:16.773359 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 238/483] 10.222.0.16 () {56 vars in 1102 bytes} [Wed Aug 24 09:08:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:08:16.778756 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7191fc72-5d2b-48ca-b065-44dd9b337116 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:16.778892 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415020f3-9c71-4b42-a728-f0d4b6979ced None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:16.779234 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415020f3-9c71-4b42-a728-f0d4b6979ced None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:16.779522 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415020f3-9c71-4b42-a728-f0d4b6979ced None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:16.781138 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 246/484] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:08:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:08:16.789189 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aad84e1-2c94-4783-b288-ea9520ea9748 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:16.789473 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aad84e1-2c94-4783-b288-ea9520ea9748 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:16.789749 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aad84e1-2c94-4783-b288-ea9520ea9748 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:16.822004 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-8aad84e1-2c94-4783-b288-ea9520ea9748 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:08:16.823204 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-8aad84e1-2c94-4783-b288-ea9520ea9748 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:16.926327 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7191fc72-5d2b-48ca-b065-44dd9b337116 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:16.945866 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 247/485] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:08:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:08:16.983621 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45aa912-69eb-4fb6-9854-b47937ac1ce3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:16.984232 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45aa912-69eb-4fb6-9854-b47937ac1ce3 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:16.984514 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45aa912-69eb-4fb6-9854-b47937ac1ce3 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:16.984786 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45aa912-69eb-4fb6-9854-b47937ac1ce3 None admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:16.985457 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45aa912-69eb-4fb6-9854-b47937ac1ce3 None admin] RBAC: Adding query filter params (name=service) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:16.985769 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45aa912-69eb-4fb6-9854-b47937ac1ce3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:16.987135 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45aa912-69eb-4fb6-9854-b47937ac1ce3 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:17.000461 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 239/486] 10.222.0.16 () {56 vars in 1103 bytes} [Wed Aug 24 09:08:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:08:17.001720 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aad84e1-2c94-4783-b288-ea9520ea9748 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:17.111410 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:17.142019 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aad84e1-2c94-4783-b288-ea9520ea9748 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:17.186757 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08af38b9-56f1-4c46-ba42-a42bb6e51fce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:17.187324 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08af38b9-56f1-4c46-ba42-a42bb6e51fce None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:17.187630 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08af38b9-56f1-4c46-ba42-a42bb6e51fce None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:17.187911 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08af38b9-56f1-4c46-ba42-a42bb6e51fce None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:17.205307 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08af38b9-56f1-4c46-ba42-a42bb6e51fce None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:17.206778 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08af38b9-56f1-4c46-ba42-a42bb6e51fce None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:17.225486 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-08af38b9-56f1-4c46-ba42-a42bb6e51fce None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:08:17.225486 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:08:17.225486 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:08:17.225486 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:08:17.225486 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:08:17.225486 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:08:17.225486 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:08:17.225486 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:08:17.225486 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:08:17.225486 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:08:17.225486 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:08:17.225486 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:08:17.225486 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:08:17.225486 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:08:17.225486 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:08:17.225486 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:08:17.226409 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:08:17.226409 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:08:17.226409 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:08:17.226409 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:08:17.226409 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:08:17.226409 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:08:17.226409 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:08:17.226409 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 24 09:08:17.226409 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:08:17.226409 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:08:17.226409 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:08:17.226409 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:08:17.226409 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:08:17.226409 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:08:17.226409 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:08:17.226409 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:08:17.226409 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:08:17.227266 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:08:17.227266 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:08:17.227266 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 24 09:08:17.227266 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 248/487] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 24 09:08:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:08:17.233704 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45aa912-69eb-4fb6-9854-b47937ac1ce3 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:17.275267 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:17.294935 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08af38b9-56f1-4c46-ba42-a42bb6e51fce None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:17.361332 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45aa912-69eb-4fb6-9854-b47937ac1ce3 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:17.411209 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365a6822-7943-4876-aa70-7153a805abfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:17.411891 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365a6822-7943-4876-aa70-7153a805abfe None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:17.412177 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365a6822-7943-4876-aa70-7153a805abfe None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:17.412456 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365a6822-7943-4876-aa70-7153a805abfe None admin] PATH_INFO: `/v3/domains` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:17.413117 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365a6822-7943-4876-aa70-7153a805abfe None admin] RBAC: Adding query filter params (name=Default) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:17.413427 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365a6822-7943-4876-aa70-7153a805abfe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:17.414815 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365a6822-7943-4876-aa70-7153a805abfe None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:17.427914 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 240/488] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Aug 24 09:08:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:17.429388 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356628fc-6aa7-4bf9-b8fe-bbf15ffb171f None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:17.429868 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356628fc-6aa7-4bf9-b8fe-bbf15ffb171f None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:17.430334 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356628fc-6aa7-4bf9-b8fe-bbf15ffb171f None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:17.432024 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 241/489] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:08:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:08:17.433951 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03e7f04-38cd-44cc-a0e3-0f755e4d9407 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:17.434430 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03e7f04-38cd-44cc-a0e3-0f755e4d9407 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:17.434847 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03e7f04-38cd-44cc-a0e3-0f755e4d9407 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:17.436512 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 242/490] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:08:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:08:17.437260 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03e7f04-38cd-44cc-a0e3-0f755e4d9407 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:17.448000 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08af38b9-56f1-4c46-ba42-a42bb6e51fce None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:17.478791 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:17.487653 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba3d249-812a-4e0e-a7d6-ca3dffb6eb56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:17.488520 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba3d249-812a-4e0e-a7d6-ca3dffb6eb56 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:17.488940 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba3d249-812a-4e0e-a7d6-ca3dffb6eb56 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:17.489556 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba3d249-812a-4e0e-a7d6-ca3dffb6eb56 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:17.501590 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba3d249-812a-4e0e-a7d6-ca3dffb6eb56 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:08:17.507525 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-1ba3d249-812a-4e0e-a7d6-ca3dffb6eb56 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 09:08:17.507525 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:08:17.507525 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:08:17.507525 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:08:17.507525 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:08:17.507525 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:08:17.507525 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:08:17.507525 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:08:17.507525 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:08:17.507525 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:08:17.507525 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:08:17.507525 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:08:17.507525 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:08:17.507525 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:08:17.507525 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:08:17.507525 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:08:17.508466 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:08:17.508466 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:08:17.508466 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:08:17.508466 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:08:17.508466 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:08:17.508466 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:08:17.508466 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:08:17.508466 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:08:17.508466 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:08:17.508466 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:08:17.508466 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:08:17.508466 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 24 09:08:17.508466 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:08:17.508466 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:08:17.508466 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:08:17.508466 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:08:17.509277 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:08:17.509277 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:08:17.509277 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:08:17.509277 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:08:17.509277 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:08:17.509277 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:08:17.509277 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:08:17.509277 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:08:17.509277 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:08:17.509277 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:08:17.509277 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 09:08:17.509277 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 24 09:08:17.509277 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 249/491] 10.222.0.16 () {56 vars in 1102 bytes} [Wed Aug 24 09:08:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 09:08:17.509953 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e448193b-5749-404e-8f27-b1a1fad79928 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:17.510021 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e448193b-5749-404e-8f27-b1a1fad79928 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:17.510271 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e448193b-5749-404e-8f27-b1a1fad79928 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:17.530310 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-e448193b-5749-404e-8f27-b1a1fad79928 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:08:17.530866 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:17.531832 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-e448193b-5749-404e-8f27-b1a1fad79928 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:17.552145 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03e7f04-38cd-44cc-a0e3-0f755e4d9407 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:17.587531 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd701a8-cfeb-4387-bf96-fa19f4d30c2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:17.588257 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd701a8-cfeb-4387-bf96-fa19f4d30c2d None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:17.588541 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd701a8-cfeb-4387-bf96-fa19f4d30c2d None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:17.588816 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd701a8-cfeb-4387-bf96-fa19f4d30c2d None admin] PATH_INFO: `/v3/users/nova` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:17.596795 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd701a8-cfeb-4387-bf96-fa19f4d30c2d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:17.598240 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd701a8-cfeb-4387-bf96-fa19f4d30c2d None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:17.607744 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 250/492] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:08:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:08:17.607744 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cc37de-5649-485d-afc0-e48674eee616 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:17.607744 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cc37de-5649-485d-afc0-e48674eee616 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:17.607744 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cc37de-5649-485d-afc0-e48674eee616 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:17.618589 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-abd701a8-cfeb-4387-bf96-fa19f4d30c2d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 09:08:17.618589 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:08:17.618589 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:08:17.618589 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:08:17.618589 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:08:17.618589 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:08:17.618589 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:08:17.618589 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:08:17.618589 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:08:17.618589 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:08:17.618589 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:08:17.618589 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:08:17.618589 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 09:08:17.618589 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 09:08:17.618589 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 09:08:17.618589 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 09:08:17.619717 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:08:17.619717 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:08:17.619717 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 09:08:17.619717 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:08:17.619717 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 09:08:17.619717 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:08:17.619717 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:08:17.619717 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:08:17.619717 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:08:17.619717 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:08:17.619717 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:08:17.619717 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Aug 24 09:08:17.619717 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:08:17.619717 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:08:17.619717 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:08:17.619717 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:08:17.619717 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:08:17.619717 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:08:17.620617 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:08:17.620617 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:08:17.620617 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 09:08:17.620617 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 09:08:17.620617 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 09:08:17.620617 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 09:08:17.620617 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 09:08:17.620617 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 09:08:17.620617 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 09:08:17.620617 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Aug 24 09:08:17.627063 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 243/493] 10.222.0.16 () {56 vars in 1093 bytes} [Wed Aug 24 09:08:17 2022] GET /identity/v3/users/nova => generated 82 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:08:17.627613 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-c2cc37de-5649-485d-afc0-e48674eee616 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:08:17.628306 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd701a8-cfeb-4387-bf96-fa19f4d30c2d None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:17.629399 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-c2cc37de-5649-485d-afc0-e48674eee616 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:17.719451 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 251/494] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:08:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:08:17.721077 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cc37de-5649-485d-afc0-e48674eee616 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:17.773562 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd701a8-cfeb-4387-bf96-fa19f4d30c2d None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:17.778400 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:17.835282 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cc37de-5649-485d-afc0-e48674eee616 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:17.868305 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94a4227-1bd0-4a05-836d-2c927ca7368c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:17.869055 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94a4227-1bd0-4a05-836d-2c927ca7368c None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:17.869444 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94a4227-1bd0-4a05-836d-2c927ca7368c None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:17.869801 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94a4227-1bd0-4a05-836d-2c927ca7368c None admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:17.870558 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94a4227-1bd0-4a05-836d-2c927ca7368c None admin] RBAC: Adding query filter params (name=service) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:17.870962 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94a4227-1bd0-4a05-836d-2c927ca7368c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:17.877112 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94a4227-1bd0-4a05-836d-2c927ca7368c None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:17.879800 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594d63d8-3942-4fe2-a7d1-111a31d4f88f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:17.880500 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594d63d8-3942-4fe2-a7d1-111a31d4f88f None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:17.880850 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594d63d8-3942-4fe2-a7d1-111a31d4f88f None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:17.881191 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594d63d8-3942-4fe2-a7d1-111a31d4f88f None admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:17.882417 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594d63d8-3942-4fe2-a7d1-111a31d4f88f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:17.882417 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594d63d8-3942-4fe2-a7d1-111a31d4f88f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:17.883776 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594d63d8-3942-4fe2-a7d1-111a31d4f88f None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:17.886207 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 244/495] 10.222.0.16 () {56 vars in 1103 bytes} [Wed Aug 24 09:08:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:17.887674 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94a4227-1bd0-4a05-836d-2c927ca7368c None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:17.892279 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:17.931784 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 252/496] 10.222.0.16 () {56 vars in 1133 bytes} [Wed Aug 24 09:08:17 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:17.931784 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594d63d8-3942-4fe2-a7d1-111a31d4f88f None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:18.021964 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94a4227-1bd0-4a05-836d-2c927ca7368c None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:18.046336 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594d63d8-3942-4fe2-a7d1-111a31d4f88f None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:18.056794 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:18.057603 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734b869c-d295-4e12-8132-45433bed4ce6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:18.058302 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734b869c-d295-4e12-8132-45433bed4ce6 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:18.059095 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734b869c-d295-4e12-8132-45433bed4ce6 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:18.059373 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734b869c-d295-4e12-8132-45433bed4ce6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:18.069660 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734b869c-d295-4e12-8132-45433bed4ce6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:08:18.075041 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-734b869c-d295-4e12-8132-45433bed4ce6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 09:08:18.075041 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:08:18.075041 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:08:18.075041 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:08:18.075041 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:08:18.075041 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:08:18.075041 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:08:18.075041 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:08:18.075041 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:08:18.075041 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:08:18.075041 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:08:18.075041 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:08:18.075041 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:08:18.075041 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:08:18.075041 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:08:18.075041 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:08:18.076050 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:08:18.076050 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:08:18.076050 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:08:18.076050 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:08:18.076050 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:08:18.076050 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:08:18.076050 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:08:18.076050 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:08:18.076050 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:08:18.076050 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:08:18.076050 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:08:18.076050 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Aug 24 09:08:18.076050 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:08:18.076050 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:08:18.076050 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:08:18.076050 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:08:18.077042 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:08:18.077042 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:08:18.077042 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:08:18.077042 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:08:18.077042 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:08:18.077042 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:08:18.077042 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:08:18.077042 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:08:18.077042 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:08:18.077042 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:08:18.077042 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 09:08:18.077042 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Aug 24 09:08:18.077042 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 245/497] 10.222.0.16 () {56 vars in 1102 bytes} [Wed Aug 24 09:08:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:08:18.077831 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734b869c-d295-4e12-8132-45433bed4ce6 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:18.089981 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5138f5-df3e-4a3a-b8c6-5f937ba2a865 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:18.090656 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5138f5-df3e-4a3a-b8c6-5f937ba2a865 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:18.091131 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5138f5-df3e-4a3a-b8c6-5f937ba2a865 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:18.091592 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5138f5-df3e-4a3a-b8c6-5f937ba2a865 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:18.102987 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5138f5-df3e-4a3a-b8c6-5f937ba2a865 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:18.104537 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5138f5-df3e-4a3a-b8c6-5f937ba2a865 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:18.120490 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-bc5138f5-df3e-4a3a-b8c6-5f937ba2a865 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:08:18.120490 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:08:18.120490 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:08:18.120490 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:08:18.120490 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:08:18.120490 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:08:18.120490 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:08:18.120490 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:08:18.120490 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:08:18.120490 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:08:18.120490 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:08:18.120490 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:08:18.120490 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:08:18.120490 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:08:18.120490 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:08:18.120490 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:08:18.122454 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:08:18.122454 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:08:18.122454 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:08:18.122454 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:08:18.122454 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:08:18.122454 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:08:18.122454 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:08:18.122454 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 24 09:08:18.122454 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:08:18.122454 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:08:18.122454 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:08:18.122454 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:08:18.122454 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:08:18.122454 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:08:18.122454 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:08:18.122454 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:08:18.122454 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:08:18.123316 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:08:18.123316 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:08:18.123316 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 24 09:08:18.123316 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 253/498] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 24 09:08:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:08:18.123316 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5138f5-df3e-4a3a-b8c6-5f937ba2a865 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:18.155536 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:18.195110 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734b869c-d295-4e12-8132-45433bed4ce6 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:18.232286 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8049c21c-a31b-4f73-a8a9-8e5bdf10fdc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:18.232898 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8049c21c-a31b-4f73-a8a9-8e5bdf10fdc1 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:18.233014 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5138f5-df3e-4a3a-b8c6-5f937ba2a865 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:18.233178 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8049c21c-a31b-4f73-a8a9-8e5bdf10fdc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:18.233451 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8049c21c-a31b-4f73-a8a9-8e5bdf10fdc1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:18.269033 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f351297e-aa38-4efc-8a91-2e1f3da9902a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:18.269758 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f351297e-aa38-4efc-8a91-2e1f3da9902a None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:18.270107 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f351297e-aa38-4efc-8a91-2e1f3da9902a None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:18.270388 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f351297e-aa38-4efc-8a91-2e1f3da9902a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:18.281302 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f351297e-aa38-4efc-8a91-2e1f3da9902a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:08:18.286468 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-f351297e-aa38-4efc-8a91-2e1f3da9902a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 09:08:18.286468 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:08:18.286468 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:08:18.286468 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:08:18.286468 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:08:18.286468 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:08:18.286468 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:08:18.286468 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:08:18.286468 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:08:18.286468 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:08:18.286468 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:08:18.286468 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:08:18.286468 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:08:18.286468 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:08:18.286468 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:08:18.286468 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:08:18.287370 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:08:18.287370 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:08:18.287370 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:08:18.287370 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:08:18.287370 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:08:18.287370 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:08:18.287370 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:08:18.287370 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:08:18.287370 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:08:18.287370 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:08:18.287370 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:08:18.287370 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 24 09:08:18.287370 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:08:18.287370 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:08:18.287370 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:08:18.287370 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:08:18.288238 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:08:18.288238 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:08:18.288238 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:08:18.288238 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:08:18.288238 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:08:18.288238 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:08:18.288238 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:08:18.288238 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:08:18.288238 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:08:18.288238 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:08:18.288238 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 09:08:18.288238 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 24 09:08:18.288238 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 254/499] 10.222.0.16 () {56 vars in 1102 bytes} [Wed Aug 24 09:08:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:08:18.288905 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f351297e-aa38-4efc-8a91-2e1f3da9902a None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:18.310208 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8049c21c-a31b-4f73-a8a9-8e5bdf10fdc1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:18.311783 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8049c21c-a31b-4f73-a8a9-8e5bdf10fdc1 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:18.314917 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:18.374623 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:18.391777 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f351297e-aa38-4efc-8a91-2e1f3da9902a None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:18.413210 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 246/500] 10.222.0.16 () {58 vars in 1282 bytes} [Wed Aug 24 09:08:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 09:08:18.414045 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8049c21c-a31b-4f73-a8a9-8e5bdf10fdc1 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:18.426259 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18481713-7a23-48fc-a838-e1f5fc4bdb50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:18.426868 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18481713-7a23-48fc-a838-e1f5fc4bdb50 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:18.427148 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18481713-7a23-48fc-a838-e1f5fc4bdb50 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:18.427421 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18481713-7a23-48fc-a838-e1f5fc4bdb50 None admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:18.428125 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18481713-7a23-48fc-a838-e1f5fc4bdb50 None admin] RBAC: Adding query filter params (name=service) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:18.428432 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18481713-7a23-48fc-a838-e1f5fc4bdb50 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:18.429805 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18481713-7a23-48fc-a838-e1f5fc4bdb50 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:18.437601 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 255/501] 10.222.0.16 () {56 vars in 1103 bytes} [Wed Aug 24 09:08:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:18.438735 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e686c36e-8426-455a-a2f0-826e80a1797e None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:18.439021 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e686c36e-8426-455a-a2f0-826e80a1797e None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:18.439300 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e686c36e-8426-455a-a2f0-826e80a1797e None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:18.443754 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 256/502] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:08:18 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:08:18.443754 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e686c36e-8426-455a-a2f0-826e80a1797e None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:18.516655 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8049c21c-a31b-4f73-a8a9-8e5bdf10fdc1 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:18.521664 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:18.540528 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e686c36e-8426-455a-a2f0-826e80a1797e None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:18.551034 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadf24f1-093e-4738-b4f3-c7dea493b493 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:18.551661 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadf24f1-093e-4738-b4f3-c7dea493b493 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:18.551943 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadf24f1-093e-4738-b4f3-c7dea493b493 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:18.552218 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadf24f1-093e-4738-b4f3-c7dea493b493 None admin] PATH_INFO: `/v3/domains` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:18.552851 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eadf24f1-093e-4738-b4f3-c7dea493b493 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:18.553229 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eadf24f1-093e-4738-b4f3-c7dea493b493 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:18.554588 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eadf24f1-093e-4738-b4f3-c7dea493b493 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:18.565689 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 247/503] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Aug 24 09:08:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:18.566487 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadf24f1-093e-4738-b4f3-c7dea493b493 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:18.575146 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a5cc1b-5431-4c2d-966d-725373747779 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:18.575778 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a5cc1b-5431-4c2d-966d-725373747779 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:18.576060 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a5cc1b-5431-4c2d-966d-725373747779 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:18.576330 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a5cc1b-5431-4c2d-966d-725373747779 None admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:18.576986 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a5cc1b-5431-4c2d-966d-725373747779 None admin] RBAC: Adding query filter params (name=service) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:18.577289 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a5cc1b-5431-4c2d-966d-725373747779 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:18.578681 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a5cc1b-5431-4c2d-966d-725373747779 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:18.586760 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 257/504] 10.222.0.16 () {56 vars in 1103 bytes} [Wed Aug 24 09:08:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:18.587522 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a5cc1b-5431-4c2d-966d-725373747779 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:18.593573 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:18.669528 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadf24f1-093e-4738-b4f3-c7dea493b493 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:18.685533 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a5cc1b-5431-4c2d-966d-725373747779 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:18.702595 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb40d7a-70cc-486c-ad85-428a12ed22be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:18.703215 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb40d7a-70cc-486c-ad85-428a12ed22be None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:18.703494 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb40d7a-70cc-486c-ad85-428a12ed22be None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:18.703799 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb40d7a-70cc-486c-ad85-428a12ed22be None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:18.715150 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb40d7a-70cc-486c-ad85-428a12ed22be None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:18.716682 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb40d7a-70cc-486c-ad85-428a12ed22be None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:18.719173 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12313ea5-41f2-43bb-ac6d-c950c9017c60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:18.719822 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12313ea5-41f2-43bb-ac6d-c950c9017c60 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:18.720095 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12313ea5-41f2-43bb-ac6d-c950c9017c60 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:18.720370 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12313ea5-41f2-43bb-ac6d-c950c9017c60 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:18.737126 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-8cb40d7a-70cc-486c-ad85-428a12ed22be None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:08:18.737126 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:08:18.737126 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:08:18.737126 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:08:18.737126 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:08:18.737126 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:08:18.737126 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:08:18.737126 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:08:18.737126 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:08:18.737126 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:08:18.737126 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:08:18.737126 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:08:18.737126 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:08:18.737126 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:08:18.737126 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:08:18.737126 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:08:18.738086 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:08:18.738086 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:08:18.738086 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:08:18.738086 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:08:18.738086 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:08:18.738086 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:08:18.738086 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:08:18.738086 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Aug 24 09:08:18.738086 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:08:18.738086 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:08:18.738086 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:08:18.738086 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:08:18.738086 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:08:18.738086 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:08:18.738086 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:08:18.738086 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:08:18.738086 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:08:18.739118 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:08:18.739118 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:08:18.739118 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Aug 24 09:08:18.739118 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 248/505] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 24 09:08:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:08:18.739506 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831f6e23-75fd-49e1-a4a2-023e18df80e7 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:18.739821 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831f6e23-75fd-49e1-a4a2-023e18df80e7 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:18.740098 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831f6e23-75fd-49e1-a4a2-023e18df80e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:18.742102 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12313ea5-41f2-43bb-ac6d-c950c9017c60 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:18.743560 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12313ea5-41f2-43bb-ac6d-c950c9017c60 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:18.746268 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:18.758172 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-12313ea5-41f2-43bb-ac6d-c950c9017c60 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:08:18.758172 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:08:18.758172 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:08:18.758172 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:08:18.758172 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:08:18.758172 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:08:18.758172 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:08:18.758172 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:08:18.758172 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:08:18.758172 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:08:18.758172 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:08:18.758172 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:08:18.758172 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:08:18.758172 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:08:18.758172 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:08:18.758172 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:08:18.765496 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:08:18.765496 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:08:18.765496 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:08:18.765496 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:08:18.765496 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:08:18.765496 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:08:18.765496 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:08:18.765496 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 24 09:08:18.765496 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:08:18.765496 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:08:18.765496 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:08:18.765496 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:08:18.765496 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:08:18.765496 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:08:18.765496 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:08:18.765496 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:08:18.765496 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:08:18.766685 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:08:18.766685 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:08:18.766685 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 24 09:08:18.766894 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-831f6e23-75fd-49e1-a4a2-023e18df80e7 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:08:18.766981 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 258/506] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 24 09:08:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:08:18.767059 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-831f6e23-75fd-49e1-a4a2-023e18df80e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:18.767138 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12313ea5-41f2-43bb-ac6d-c950c9017c60 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:18.832043 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 249/507] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:08:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:08:18.832806 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831f6e23-75fd-49e1-a4a2-023e18df80e7 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:18.840904 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:18.867909 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12313ea5-41f2-43bb-ac6d-c950c9017c60 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:18.903674 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58321a46-63d0-4341-b2a1-0015d47eacb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:18.904301 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58321a46-63d0-4341-b2a1-0015d47eacb6 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:18.904589 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58321a46-63d0-4341-b2a1-0015d47eacb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:18.904867 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58321a46-63d0-4341-b2a1-0015d47eacb6 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:18.912634 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58321a46-63d0-4341-b2a1-0015d47eacb6 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:18.914140 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58321a46-63d0-4341-b2a1-0015d47eacb6 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:18.927535 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-58321a46-63d0-4341-b2a1-0015d47eacb6 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 24 09:08:18.927535 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:08:18.927535 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:08:18.927535 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:08:18.927535 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:08:18.927535 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:08:18.927535 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:08:18.927535 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:08:18.927535 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:08:18.927535 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:08:18.927535 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:08:18.927535 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:08:18.927535 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 09:08:18.927535 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 09:08:18.927535 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 09:08:18.927535 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 09:08:18.928589 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:08:18.928589 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:08:18.928589 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 09:08:18.928589 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:08:18.928589 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 09:08:18.928589 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:08:18.928589 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:08:18.928589 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:08:18.928589 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:08:18.928589 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:08:18.928589 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:08:18.928589 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 24 09:08:18.928589 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:08:18.928589 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:08:18.928589 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:08:18.928589 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:08:18.928589 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:08:18.928589 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:08:18.929496 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:08:18.929496 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:08:18.929496 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 09:08:18.929496 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 09:08:18.929496 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 09:08:18.929496 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 09:08:18.929496 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 09:08:18.929496 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 09:08:18.929496 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 24 09:08:18.929496 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 24 09:08:18.929496 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 259/508] 10.222.0.16 () {56 vars in 1099 bytes} [Wed Aug 24 09:08:18 2022] GET /identity/v3/users/glance => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:08:18.929496 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58321a46-63d0-4341-b2a1-0015d47eacb6 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:18.940452 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831f6e23-75fd-49e1-a4a2-023e18df80e7 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:18.976034 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721768e6-b915-4ea8-a5d0-73213e90da3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:18.976661 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721768e6-b915-4ea8-a5d0-73213e90da3c None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:18.976942 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721768e6-b915-4ea8-a5d0-73213e90da3c None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:18.977213 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721768e6-b915-4ea8-a5d0-73213e90da3c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:18.987826 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721768e6-b915-4ea8-a5d0-73213e90da3c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:18.989278 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721768e6-b915-4ea8-a5d0-73213e90da3c None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:18.989433 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:19.004168 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-721768e6-b915-4ea8-a5d0-73213e90da3c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:08:19.004168 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:08:19.004168 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:08:19.004168 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:08:19.004168 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:08:19.004168 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:08:19.004168 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:08:19.004168 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:08:19.004168 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:08:19.004168 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:08:19.004168 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:08:19.004168 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:08:19.004168 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:08:19.004168 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:08:19.004168 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:08:19.004168 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:08:19.005325 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:08:19.005325 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:08:19.005325 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:08:19.005325 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:08:19.005325 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:08:19.005325 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:08:19.005325 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:08:19.005325 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Aug 24 09:08:19.005325 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:08:19.005325 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:08:19.005325 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:08:19.005325 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:08:19.005325 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:08:19.005325 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:08:19.005325 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:08:19.005325 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:08:19.005325 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:08:19.006172 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:08:19.006172 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:08:19.006172 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Aug 24 09:08:19.010009 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 250/509] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 24 09:08:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:08:19.010862 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721768e6-b915-4ea8-a5d0-73213e90da3c None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:19.035099 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58321a46-63d0-4341-b2a1-0015d47eacb6 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:19.071208 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:19.071811 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458c3756-ffad-47ce-bd0b-0908328cadb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:19.072419 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458c3756-ffad-47ce-bd0b-0908328cadb1 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:19.072705 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458c3756-ffad-47ce-bd0b-0908328cadb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:19.073013 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458c3756-ffad-47ce-bd0b-0908328cadb1 None admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:19.073656 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458c3756-ffad-47ce-bd0b-0908328cadb1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:19.073993 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458c3756-ffad-47ce-bd0b-0908328cadb1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:19.075372 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458c3756-ffad-47ce-bd0b-0908328cadb1 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:19.093350 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 260/510] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Aug 24 09:08:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:19.093350 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458c3756-ffad-47ce-bd0b-0908328cadb1 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:19.116417 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721768e6-b915-4ea8-a5d0-73213e90da3c None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:19.149356 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ed8aa3-5775-4e55-94fd-7a25ecfb5df4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:19.149954 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ed8aa3-5775-4e55-94fd-7a25ecfb5df4 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:19.155354 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ed8aa3-5775-4e55-94fd-7a25ecfb5df4 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:19.155354 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ed8aa3-5775-4e55-94fd-7a25ecfb5df4 None admin] PATH_INFO: `/v3/domains` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:19.155354 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ed8aa3-5775-4e55-94fd-7a25ecfb5df4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:19.155354 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ed8aa3-5775-4e55-94fd-7a25ecfb5df4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:19.155354 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ed8aa3-5775-4e55-94fd-7a25ecfb5df4 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:19.165082 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 251/511] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Aug 24 09:08:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:19.165877 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ed8aa3-5775-4e55-94fd-7a25ecfb5df4 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:19.195394 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458c3756-ffad-47ce-bd0b-0908328cadb1 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:19.241409 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:19.259721 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61104f21-9317-49fb-8cd3-84ef22042549 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:19.260327 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61104f21-9317-49fb-8cd3-84ef22042549 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:19.260608 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61104f21-9317-49fb-8cd3-84ef22042549 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:19.260879 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61104f21-9317-49fb-8cd3-84ef22042549 None admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:19.261635 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61104f21-9317-49fb-8cd3-84ef22042549 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:19.261942 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61104f21-9317-49fb-8cd3-84ef22042549 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:19.263365 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61104f21-9317-49fb-8cd3-84ef22042549 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:19.264122 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ed8aa3-5775-4e55-94fd-7a25ecfb5df4 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:19.280861 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 261/512] 10.222.0.16 () {56 vars in 1137 bytes} [Wed Aug 24 09:08:19 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:19.281658 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61104f21-9317-49fb-8cd3-84ef22042549 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:19.281901 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:19.299226 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf0d0fa-630b-43c6-889d-a5c7a1a38fa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:19.299956 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf0d0fa-630b-43c6-889d-a5c7a1a38fa1 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:19.300242 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf0d0fa-630b-43c6-889d-a5c7a1a38fa1 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:19.300518 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf0d0fa-630b-43c6-889d-a5c7a1a38fa1 None admin] PATH_INFO: `/v3/domains` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:19.301163 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf0d0fa-630b-43c6-889d-a5c7a1a38fa1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:19.301469 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf0d0fa-630b-43c6-889d-a5c7a1a38fa1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:19.302868 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf0d0fa-630b-43c6-889d-a5c7a1a38fa1 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:19.314352 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 252/513] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Aug 24 09:08:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:19.315146 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf0d0fa-630b-43c6-889d-a5c7a1a38fa1 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:19.389016 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61104f21-9317-49fb-8cd3-84ef22042549 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:19.417104 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf0d0fa-630b-43c6-889d-a5c7a1a38fa1 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:19.423348 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d20c6bd-8e12-4cc0-8270-97e5ea0c8d5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:19.424013 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d20c6bd-8e12-4cc0-8270-97e5ea0c8d5d None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:19.424303 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d20c6bd-8e12-4cc0-8270-97e5ea0c8d5d None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:19.424578 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d20c6bd-8e12-4cc0-8270-97e5ea0c8d5d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:19.431246 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d20c6bd-8e12-4cc0-8270-97e5ea0c8d5d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:19.432775 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d20c6bd-8e12-4cc0-8270-97e5ea0c8d5d None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:19.445619 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-7d20c6bd-8e12-4cc0-8270-97e5ea0c8d5d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:08:19.445619 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:08:19.445619 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:08:19.445619 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:08:19.445619 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:08:19.445619 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:08:19.445619 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:08:19.445619 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:08:19.445619 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:08:19.445619 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:08:19.445619 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:08:19.445619 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:08:19.445619 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 09:08:19.445619 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 09:08:19.445619 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 09:08:19.446480 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 09:08:19.446480 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:08:19.446480 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:08:19.446480 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:08:19.446480 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:08:19.446480 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:08:19.446480 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:08:19.446480 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:08:19.446480 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 24 09:08:19.446480 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:08:19.446480 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:08:19.446480 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:08:19.446480 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:08:19.446480 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:08:19.446480 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:08:19.446480 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:08:19.446480 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:08:19.447370 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 09:08:19.447370 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 09:08:19.447370 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 09:08:19.447370 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 09:08:19.447370 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 09:08:19.447370 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 09:08:19.447370 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:08:19.447370 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 24 09:08:19.447370 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 262/514] 10.222.0.16 () {56 vars in 1111 bytes} [Wed Aug 24 09:08:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:08:19.447908 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d20c6bd-8e12-4cc0-8270-97e5ea0c8d5d None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:19.456119 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5356ce-b716-45d0-8855-333e1df0d1a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:19.456744 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5356ce-b716-45d0-8855-333e1df0d1a9 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:19.457023 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5356ce-b716-45d0-8855-333e1df0d1a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:19.457322 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5356ce-b716-45d0-8855-333e1df0d1a9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:19.468600 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5356ce-b716-45d0-8855-333e1df0d1a9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:19.470046 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5356ce-b716-45d0-8855-333e1df0d1a9 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:19.481182 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:19.484215 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-5d5356ce-b716-45d0-8855-333e1df0d1a9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:08:19.484215 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:08:19.484215 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:08:19.484215 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:08:19.484215 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:08:19.484215 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:08:19.484215 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:08:19.484215 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:08:19.484215 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:08:19.484215 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:08:19.484215 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:08:19.484215 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:08:19.484215 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:08:19.484215 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:08:19.484215 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:08:19.484215 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:08:19.485290 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:08:19.485290 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:08:19.485290 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:08:19.485290 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:08:19.485290 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:08:19.485290 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:08:19.485290 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:08:19.485290 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Aug 24 09:08:19.485290 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:08:19.485290 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:08:19.485290 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:08:19.485290 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:08:19.485290 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:08:19.485290 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:08:19.485290 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:08:19.485290 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:08:19.485290 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:08:19.486462 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:08:19.486462 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:08:19.486462 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Aug 24 09:08:19.486462 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 253/515] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 24 09:08:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:08:19.486462 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5356ce-b716-45d0-8855-333e1df0d1a9 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:19.519453 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:19.547091 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d20c6bd-8e12-4cc0-8270-97e5ea0c8d5d None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:19.581136 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a109911-595e-487e-9a72-539e84ecb782 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:19.581762 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a109911-595e-487e-9a72-539e84ecb782 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:19.582040 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a109911-595e-487e-9a72-539e84ecb782 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:19.582311 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a109911-595e-487e-9a72-539e84ecb782 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:19.588875 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5356ce-b716-45d0-8855-333e1df0d1a9 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:19.589144 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a109911-595e-487e-9a72-539e84ecb782 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:19.590604 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a109911-595e-487e-9a72-539e84ecb782 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:19.601686 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-8a109911-595e-487e-9a72-539e84ecb782 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 09:08:19.601686 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:08:19.601686 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:08:19.601686 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:08:19.601686 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:08:19.601686 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:08:19.601686 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:08:19.601686 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:08:19.601686 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:08:19.601686 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:08:19.601686 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:08:19.601686 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:08:19.601686 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 09:08:19.601686 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 09:08:19.601686 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 09:08:19.601686 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 09:08:19.602620 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:08:19.602620 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:08:19.602620 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 09:08:19.602620 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:08:19.602620 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 09:08:19.602620 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:08:19.602620 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:08:19.602620 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:08:19.602620 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:08:19.602620 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:08:19.602620 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:08:19.602620 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 24 09:08:19.602620 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:08:19.602620 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:08:19.602620 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:08:19.602620 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:08:19.602620 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:08:19.602620 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:08:19.603527 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:08:19.603527 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:08:19.603527 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 09:08:19.603527 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 09:08:19.603527 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 09:08:19.603527 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 09:08:19.603527 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 09:08:19.603527 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 09:08:19.603527 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 09:08:19.603527 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 24 09:08:19.603527 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 263/516] 10.222.0.16 () {56 vars in 1102 bytes} [Wed Aug 24 09:08:19 2022] GET /identity/v3/users/neutron => generated 85 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:08:19.604154 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a109911-595e-487e-9a72-539e84ecb782 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:19.627260 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8e9a3d-ef0f-41cf-b956-bf5565ff4316 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:19.627894 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8e9a3d-ef0f-41cf-b956-bf5565ff4316 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:19.628174 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8e9a3d-ef0f-41cf-b956-bf5565ff4316 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:19.628441 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8e9a3d-ef0f-41cf-b956-bf5565ff4316 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:19.694674 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8e9a3d-ef0f-41cf-b956-bf5565ff4316 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:19.695116 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:19.696107 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8e9a3d-ef0f-41cf-b956-bf5565ff4316 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:19.701829 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a109911-595e-487e-9a72-539e84ecb782 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:19.725071 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:19.736424 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9668bde0-727a-4ca9-a44e-7680d27b5ffb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:19.737040 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9668bde0-727a-4ca9-a44e-7680d27b5ffb None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:19.737322 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9668bde0-727a-4ca9-a44e-7680d27b5ffb None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:19.737599 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9668bde0-727a-4ca9-a44e-7680d27b5ffb None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:19.744626 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9668bde0-727a-4ca9-a44e-7680d27b5ffb None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:19.746668 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9668bde0-727a-4ca9-a44e-7680d27b5ffb None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:19.757553 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-9668bde0-727a-4ca9-a44e-7680d27b5ffb None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 09:08:19.757553 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:08:19.757553 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:08:19.757553 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:08:19.757553 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:08:19.757553 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:08:19.757553 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:08:19.757553 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:08:19.757553 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:08:19.757553 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:08:19.757553 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:08:19.757553 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:08:19.757553 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 09:08:19.757553 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 09:08:19.757553 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 09:08:19.757553 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 09:08:19.758492 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:08:19.758492 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:08:19.758492 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 09:08:19.758492 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:08:19.758492 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 09:08:19.758492 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:08:19.758492 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:08:19.758492 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:08:19.758492 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:08:19.758492 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:08:19.758492 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:08:19.758492 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 24 09:08:19.758492 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:08:19.758492 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:08:19.758492 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:08:19.758492 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:08:19.758492 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:08:19.758492 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:08:19.759376 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:08:19.759376 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:08:19.759376 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 09:08:19.759376 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 09:08:19.759376 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 09:08:19.759376 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 09:08:19.759376 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 09:08:19.759376 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 09:08:19.759376 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 09:08:19.759376 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 24 09:08:19.763371 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 264/517] 10.222.0.16 () {56 vars in 1099 bytes} [Wed Aug 24 09:08:19 2022] GET /identity/v3/users/cinder => generated 84 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:08:19.764072 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9668bde0-727a-4ca9-a44e-7680d27b5ffb None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:19.795764 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 254/518] 10.222.0.16 () {58 vars in 1282 bytes} [Wed Aug 24 09:08:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 09:08:19.796594 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8e9a3d-ef0f-41cf-b956-bf5565ff4316 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:19.862515 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9668bde0-727a-4ca9-a44e-7680d27b5ffb None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:19.893243 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8e9a3d-ef0f-41cf-b956-bf5565ff4316 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:19.895532 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b3de43-d194-4a2a-b148-3f4fe621c4df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:19.896159 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b3de43-d194-4a2a-b148-3f4fe621c4df None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:19.896443 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b3de43-d194-4a2a-b148-3f4fe621c4df None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:19.896718 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b3de43-d194-4a2a-b148-3f4fe621c4df None admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:19.897406 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b3de43-d194-4a2a-b148-3f4fe621c4df None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:19.897713 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b3de43-d194-4a2a-b148-3f4fe621c4df None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:19.899146 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b3de43-d194-4a2a-b148-3f4fe621c4df None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:19.916566 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 265/519] 10.222.0.16 () {56 vars in 1148 bytes} [Wed Aug 24 09:08:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:19.917443 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b3de43-d194-4a2a-b148-3f4fe621c4df None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:19.922676 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:19.928543 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d684e4a-2c48-4e26-99f7-5f6363d32fa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:19.929148 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d684e4a-2c48-4e26-99f7-5f6363d32fa4 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:19.932789 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d684e4a-2c48-4e26-99f7-5f6363d32fa4 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:19.932789 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d684e4a-2c48-4e26-99f7-5f6363d32fa4 None admin] PATH_INFO: `/v3/domains` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:19.932789 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d684e4a-2c48-4e26-99f7-5f6363d32fa4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:19.932789 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d684e4a-2c48-4e26-99f7-5f6363d32fa4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:19.932789 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d684e4a-2c48-4e26-99f7-5f6363d32fa4 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:19.933484 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:19.943122 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 255/520] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Aug 24 09:08:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:19.943946 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d684e4a-2c48-4e26-99f7-5f6363d32fa4 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:20.017181 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b3de43-d194-4a2a-b148-3f4fe621c4df None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:20.041795 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d684e4a-2c48-4e26-99f7-5f6363d32fa4 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:20.053938 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3931c45-0e1e-4515-9907-42522ac4d9ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:20.054531 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3931c45-0e1e-4515-9907-42522ac4d9ef None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:20.054805 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3931c45-0e1e-4515-9907-42522ac4d9ef None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:20.055070 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3931c45-0e1e-4515-9907-42522ac4d9ef None admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:20.055944 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3931c45-0e1e-4515-9907-42522ac4d9ef None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:20.056247 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3931c45-0e1e-4515-9907-42522ac4d9ef None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:20.057657 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3931c45-0e1e-4515-9907-42522ac4d9ef None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:20.074144 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 266/521] 10.222.0.16 () {56 vars in 1139 bytes} [Wed Aug 24 09:08:19 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:20.074926 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3931c45-0e1e-4515-9907-42522ac4d9ef None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:20.075421 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdaf9eea-5f87-4370-a02b-3b9e90ed5fd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:20.076057 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaf9eea-5f87-4370-a02b-3b9e90ed5fd0 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:20.076345 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaf9eea-5f87-4370-a02b-3b9e90ed5fd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:20.076620 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaf9eea-5f87-4370-a02b-3b9e90ed5fd0 None admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:20.077466 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdaf9eea-5f87-4370-a02b-3b9e90ed5fd0 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:20.077801 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdaf9eea-5f87-4370-a02b-3b9e90ed5fd0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:20.079494 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdaf9eea-5f87-4370-a02b-3b9e90ed5fd0 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:20.116201 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 256/522] 10.222.0.16 () {56 vars in 1137 bytes} [Wed Aug 24 09:08:19 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:20.117016 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdaf9eea-5f87-4370-a02b-3b9e90ed5fd0 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:20.150406 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:20.166208 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:20.179237 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3931c45-0e1e-4515-9907-42522ac4d9ef None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:20.213291 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52fa508-1b3c-49a0-8ed4-72839aa72f95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:20.213924 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52fa508-1b3c-49a0-8ed4-72839aa72f95 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:20.214191 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52fa508-1b3c-49a0-8ed4-72839aa72f95 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:20.215113 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52fa508-1b3c-49a0-8ed4-72839aa72f95 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:20.215982 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdaf9eea-5f87-4370-a02b-3b9e90ed5fd0 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:20.225197 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52fa508-1b3c-49a0-8ed4-72839aa72f95 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:20.226628 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52fa508-1b3c-49a0-8ed4-72839aa72f95 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:20.240972 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-c52fa508-1b3c-49a0-8ed4-72839aa72f95 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:08:20.240972 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:08:20.240972 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:08:20.240972 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:08:20.240972 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:08:20.240972 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:08:20.240972 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:08:20.240972 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:08:20.240972 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:08:20.240972 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:08:20.240972 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:08:20.240972 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:08:20.240972 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:08:20.240972 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:08:20.240972 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:08:20.240972 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:08:20.241875 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:08:20.241875 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:08:20.241875 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:08:20.241875 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:08:20.241875 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:08:20.241875 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:08:20.241875 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:08:20.241875 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 24 09:08:20.241875 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:08:20.241875 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:08:20.241875 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:08:20.241875 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:08:20.241875 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:08:20.241875 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:08:20.241875 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:08:20.241875 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:08:20.241875 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:08:20.242752 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:08:20.242752 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:08:20.242752 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 24 09:08:20.242752 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 267/523] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 24 09:08:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:08:20.243054 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52fa508-1b3c-49a0-8ed4-72839aa72f95 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:20.254658 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09536a07-6f52-460b-b515-9237669c1e28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:20.255267 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09536a07-6f52-460b-b515-9237669c1e28 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:20.255551 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09536a07-6f52-460b-b515-9237669c1e28 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:20.255857 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09536a07-6f52-460b-b515-9237669c1e28 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:20.256585 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09536a07-6f52-460b-b515-9237669c1e28 None admin] RBAC: Adding query filter params (role.id=67fd70a7d9794f7fb8faebffdec46dd2, scope.project.id=c467e88e2fb0491bacefcf6d2ce326d9, user.id=1f055f7dc51d43bda9b315ca67f11d58) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:20.256900 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09536a07-6f52-460b-b515-9237669c1e28 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:20.258338 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09536a07-6f52-460b-b515-9237669c1e28 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:20.264429 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 257/524] 10.222.0.16 () {56 vars in 1374 bytes} [Wed Aug 24 09:08:20 2022] GET /identity/v3/role_assignments?user.id=1f055f7dc51d43bda9b315ca67f11d58&scope.project.id=c467e88e2fb0491bacefcf6d2ce326d9&role.id=67fd70a7d9794f7fb8faebffdec46dd2 => generated 259 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:20.265208 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09536a07-6f52-460b-b515-9237669c1e28 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:20.353032 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52fa508-1b3c-49a0-8ed4-72839aa72f95 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:20.378773 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:20.390146 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ff49b8-ec0d-467c-9364-6d47bbeee9ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:20.390770 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ff49b8-ec0d-467c-9364-6d47bbeee9ab None admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:20.392483 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ff49b8-ec0d-467c-9364-6d47bbeee9ab None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:20.392483 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ff49b8-ec0d-467c-9364-6d47bbeee9ab None admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:20.392483 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ff49b8-ec0d-467c-9364-6d47bbeee9ab None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:20.393571 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ff49b8-ec0d-467c-9364-6d47bbeee9ab None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:20.400014 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09536a07-6f52-460b-b515-9237669c1e28 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:20.415636 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:20.437571 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404c9e29-12b6-4e30-b6b2-219b988a7648 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:20.438331 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404c9e29-12b6-4e30-b6b2-219b988a7648 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:20.438615 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404c9e29-12b6-4e30-b6b2-219b988a7648 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:20.438883 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404c9e29-12b6-4e30-b6b2-219b988a7648 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:20.480413 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 268/525] 10.222.0.16 () {60 vars in 1132 bytes} [Wed Aug 24 09:08:20 2022] POST /identity/v3/users => generated 247 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:08:20.481380 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ff49b8-ec0d-467c-9364-6d47bbeee9ab None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:20.525640 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404c9e29-12b6-4e30-b6b2-219b988a7648 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:20.529437 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404c9e29-12b6-4e30-b6b2-219b988a7648 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:20.609740 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ff49b8-ec0d-467c-9364-6d47bbeee9ab None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:20.651156 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a424d2f6-820f-4152-9504-b2519362772c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:20.653496 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a424d2f6-820f-4152-9504-b2519362772c None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:20.653496 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a424d2f6-820f-4152-9504-b2519362772c None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:20.653496 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a424d2f6-820f-4152-9504-b2519362772c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:20.668945 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 258/526] 10.222.0.16 () {58 vars in 1282 bytes} [Wed Aug 24 09:08:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 09:08:20.669544 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404c9e29-12b6-4e30-b6b2-219b988a7648 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:20.685998 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:20.730543 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:20.756195 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a424d2f6-820f-4152-9504-b2519362772c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:20.757978 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a424d2f6-820f-4152-9504-b2519362772c None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:20.786756 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404c9e29-12b6-4e30-b6b2-219b988a7648 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:20.834675 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2447c9aa-ae15-4e5b-93e0-12ecceb7fde7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:20.836056 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2447c9aa-ae15-4e5b-93e0-12ecceb7fde7 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:20.836533 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2447c9aa-ae15-4e5b-93e0-12ecceb7fde7 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:20.836991 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2447c9aa-ae15-4e5b-93e0-12ecceb7fde7 None admin] PATH_INFO: `/v3/domains` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:20.838076 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2447c9aa-ae15-4e5b-93e0-12ecceb7fde7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:20.838596 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2447c9aa-ae15-4e5b-93e0-12ecceb7fde7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:20.841628 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2447c9aa-ae15-4e5b-93e0-12ecceb7fde7 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:20.857223 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 259/527] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Aug 24 09:08:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:20.858364 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2447c9aa-ae15-4e5b-93e0-12ecceb7fde7 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:20.880370 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 269/528] 10.222.0.16 () {58 vars in 1282 bytes} [Wed Aug 24 09:08:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 09:08:20.881614 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a424d2f6-820f-4152-9504-b2519362772c None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:20.933060 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:20.966973 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2447c9aa-ae15-4e5b-93e0-12ecceb7fde7 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:20.992287 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:21.014623 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a424d2f6-820f-4152-9504-b2519362772c None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:21.015006 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3f560a-bffd-40ad-bef9-7d3c0fe918e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:21.015638 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3f560a-bffd-40ad-bef9-7d3c0fe918e1 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:21.015925 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3f560a-bffd-40ad-bef9-7d3c0fe918e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:21.016196 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3f560a-bffd-40ad-bef9-7d3c0fe918e1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:21.029741 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3f560a-bffd-40ad-bef9-7d3c0fe918e1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:21.031407 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3f560a-bffd-40ad-bef9-7d3c0fe918e1 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:21.050813 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-3a3f560a-bffd-40ad-bef9-7d3c0fe918e1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:08:21.050813 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:08:21.050813 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:08:21.050813 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:08:21.050813 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:08:21.050813 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:08:21.050813 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:08:21.050813 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:08:21.050813 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:08:21.050813 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:08:21.050813 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:08:21.050813 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:08:21.050813 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:08:21.050813 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:08:21.050813 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:08:21.050813 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:08:21.051947 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:08:21.051947 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:08:21.051947 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:08:21.051947 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:08:21.051947 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:08:21.051947 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:08:21.051947 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:08:21.051947 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Aug 24 09:08:21.051947 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:08:21.051947 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:08:21.051947 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:08:21.051947 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:08:21.051947 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:08:21.051947 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:08:21.051947 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:08:21.051947 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:08:21.051947 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:08:21.052820 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:08:21.052820 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:08:21.052820 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Aug 24 09:08:21.057468 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 260/529] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 24 09:08:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:08:21.058870 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3f560a-bffd-40ad-bef9-7d3c0fe918e1 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:21.062483 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ec95c9-dbd6-4dc1-8576-55aa9f996f35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:21.064193 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ec95c9-dbd6-4dc1-8576-55aa9f996f35 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:21.064699 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ec95c9-dbd6-4dc1-8576-55aa9f996f35 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:21.065233 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ec95c9-dbd6-4dc1-8576-55aa9f996f35 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:21.073550 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ec95c9-dbd6-4dc1-8576-55aa9f996f35 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:21.075745 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ec95c9-dbd6-4dc1-8576-55aa9f996f35 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:21.091646 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-33ec95c9-dbd6-4dc1-8576-55aa9f996f35 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:08:21.091646 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:08:21.091646 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:08:21.091646 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:08:21.091646 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:08:21.091646 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:08:21.091646 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:08:21.091646 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:08:21.091646 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:08:21.091646 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:08:21.091646 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:08:21.091646 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:08:21.091646 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 09:08:21.091646 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 09:08:21.091646 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 09:08:21.092548 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 09:08:21.092548 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:08:21.092548 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:08:21.092548 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:08:21.092548 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:08:21.092548 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:08:21.092548 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:08:21.092548 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:08:21.092548 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 24 09:08:21.092548 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:08:21.092548 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:08:21.092548 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:08:21.092548 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:08:21.092548 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:08:21.092548 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:08:21.092548 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:08:21.092548 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:08:21.093397 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 09:08:21.093397 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 09:08:21.093397 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 09:08:21.093397 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 09:08:21.093397 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 09:08:21.093397 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 09:08:21.093397 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:08:21.093397 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 24 09:08:21.096658 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 270/530] 10.222.0.16 () {56 vars in 1111 bytes} [Wed Aug 24 09:08:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:08:21.097710 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ec95c9-dbd6-4dc1-8576-55aa9f996f35 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:21.178104 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3f560a-bffd-40ad-bef9-7d3c0fe918e1 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:21.203886 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:21.226516 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b58ca7-708b-4e91-a516-449ec83ed727 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:21.228215 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b58ca7-708b-4e91-a516-449ec83ed727 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:21.229059 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b58ca7-708b-4e91-a516-449ec83ed727 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:21.229563 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b58ca7-708b-4e91-a516-449ec83ed727 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:21.237278 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ec95c9-dbd6-4dc1-8576-55aa9f996f35 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:21.247433 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b58ca7-708b-4e91-a516-449ec83ed727 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:21.249419 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b58ca7-708b-4e91-a516-449ec83ed727 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:21.265566 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-69b58ca7-708b-4e91-a516-449ec83ed727 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:08:21.265566 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:08:21.265566 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:08:21.265566 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:08:21.265566 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:08:21.265566 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:08:21.265566 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:08:21.265566 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:08:21.265566 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:08:21.265566 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:08:21.265566 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:08:21.265566 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:08:21.265566 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:08:21.265566 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:08:21.265566 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:08:21.265566 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:08:21.266725 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:08:21.266725 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:08:21.266725 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:08:21.266725 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:08:21.266725 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:08:21.266725 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:08:21.266725 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:08:21.266725 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Aug 24 09:08:21.266725 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:08:21.266725 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:08:21.266725 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:08:21.266725 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:08:21.266725 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:08:21.266725 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:08:21.266725 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:08:21.266725 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:08:21.266725 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:08:21.267858 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:08:21.267858 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:08:21.267858 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Aug 24 09:08:21.269578 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 261/531] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 24 09:08:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:08:21.270778 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b58ca7-708b-4e91-a516-449ec83ed727 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:21.276108 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12560d8c-e63e-4bdb-8ae4-6f86fddc9726 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:21.276581 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12560d8c-e63e-4bdb-8ae4-6f86fddc9726 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:21.277464 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12560d8c-e63e-4bdb-8ae4-6f86fddc9726 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:21.277814 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12560d8c-e63e-4bdb-8ae4-6f86fddc9726 None admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:21.278536 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12560d8c-e63e-4bdb-8ae4-6f86fddc9726 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:21.278910 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12560d8c-e63e-4bdb-8ae4-6f86fddc9726 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:21.280390 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12560d8c-e63e-4bdb-8ae4-6f86fddc9726 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:21.289128 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:21.293448 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 271/532] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Aug 24 09:08:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:21.294348 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12560d8c-e63e-4bdb-8ae4-6f86fddc9726 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:21.393257 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b58ca7-708b-4e91-a516-449ec83ed727 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:21.407365 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12560d8c-e63e-4bdb-8ae4-6f86fddc9726 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:21.432521 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499b4911-6bce-4f5c-bee0-41e6aaad9478 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:21.433849 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499b4911-6bce-4f5c-bee0-41e6aaad9478 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:21.434312 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499b4911-6bce-4f5c-bee0-41e6aaad9478 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:21.434945 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499b4911-6bce-4f5c-bee0-41e6aaad9478 None admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:21.435841 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499b4911-6bce-4f5c-bee0-41e6aaad9478 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:21.436356 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499b4911-6bce-4f5c-bee0-41e6aaad9478 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:21.438166 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499b4911-6bce-4f5c-bee0-41e6aaad9478 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:21.442822 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a66738-d219-4ce4-bee7-ad5071280afd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:21.443739 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a66738-d219-4ce4-bee7-ad5071280afd None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:21.443853 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a66738-d219-4ce4-bee7-ad5071280afd None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:21.444182 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a66738-d219-4ce4-bee7-ad5071280afd None admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:21.444905 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a66738-d219-4ce4-bee7-ad5071280afd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:21.445290 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a66738-d219-4ce4-bee7-ad5071280afd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:21.446747 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a66738-d219-4ce4-bee7-ad5071280afd None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:21.459613 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 272/533] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Aug 24 09:08:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:21.460398 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 262/534] 10.222.0.16 () {56 vars in 1148 bytes} [Wed Aug 24 09:08:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:21.460822 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a66738-d219-4ce4-bee7-ad5071280afd None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:21.465406 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499b4911-6bce-4f5c-bee0-41e6aaad9478 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:21.505058 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:21.521289 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:21.563062 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a66738-d219-4ce4-bee7-ad5071280afd None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:21.580645 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499b4911-6bce-4f5c-bee0-41e6aaad9478 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:21.599321 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb34bcd1-c48d-4357-95fd-e797fd8879f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:21.600249 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb34bcd1-c48d-4357-95fd-e797fd8879f2 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:21.600585 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb34bcd1-c48d-4357-95fd-e797fd8879f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:21.600923 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb34bcd1-c48d-4357-95fd-e797fd8879f2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:21.609718 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb34bcd1-c48d-4357-95fd-e797fd8879f2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:21.611188 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb34bcd1-c48d-4357-95fd-e797fd8879f2 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:21.625141 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51c498f-c126-4f4e-a459-cff689ecf93d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:21.625808 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51c498f-c126-4f4e-a459-cff689ecf93d None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:21.626089 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51c498f-c126-4f4e-a459-cff689ecf93d None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:21.626363 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51c498f-c126-4f4e-a459-cff689ecf93d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:21.628510 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-cb34bcd1-c48d-4357-95fd-e797fd8879f2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:08:21.628510 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:08:21.628510 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:08:21.628510 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:08:21.628510 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:08:21.628510 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:08:21.628510 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:08:21.628510 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:08:21.628510 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:08:21.628510 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:08:21.628510 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:08:21.628510 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:08:21.628510 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 09:08:21.628510 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 09:08:21.628510 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 09:08:21.629342 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 09:08:21.629342 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:08:21.629342 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:08:21.629342 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:08:21.629342 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:08:21.629342 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:08:21.629342 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:08:21.629342 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:08:21.629342 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 24 09:08:21.629342 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:08:21.629342 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:08:21.629342 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:08:21.629342 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:08:21.629342 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:08:21.629342 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:08:21.629342 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:08:21.629342 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:08:21.630196 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 09:08:21.630196 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 09:08:21.630196 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 09:08:21.630196 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 09:08:21.630196 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 09:08:21.630196 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 09:08:21.630196 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:08:21.630196 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 24 09:08:21.633202 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 273/535] 10.222.0.16 () {56 vars in 1111 bytes} [Wed Aug 24 09:08:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:08:21.634277 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb34bcd1-c48d-4357-95fd-e797fd8879f2 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:21.635941 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51c498f-c126-4f4e-a459-cff689ecf93d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:21.638362 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51c498f-c126-4f4e-a459-cff689ecf93d None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:21.651069 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-f51c498f-c126-4f4e-a459-cff689ecf93d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:08:21.651069 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:08:21.651069 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:08:21.651069 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:08:21.651069 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:08:21.651069 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:08:21.651069 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:08:21.651069 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:08:21.651069 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:08:21.651069 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:08:21.651069 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:08:21.651069 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:08:21.651069 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 09:08:21.651069 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 09:08:21.651069 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 09:08:21.652593 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 09:08:21.652593 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:08:21.652593 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:08:21.652593 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:08:21.652593 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:08:21.652593 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:08:21.652593 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:08:21.652593 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:08:21.652593 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Aug 24 09:08:21.652593 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:08:21.652593 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:08:21.652593 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:08:21.652593 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:08:21.652593 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:08:21.652593 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:08:21.652593 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:08:21.652593 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:08:21.653444 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 09:08:21.653444 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 09:08:21.653444 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 09:08:21.653444 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 09:08:21.653444 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 09:08:21.653444 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 09:08:21.653444 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:08:21.653444 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Aug 24 09:08:21.653444 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 263/536] 10.222.0.16 () {56 vars in 1111 bytes} [Wed Aug 24 09:08:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:08:21.653909 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51c498f-c126-4f4e-a459-cff689ecf93d None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:21.761023 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:21.767998 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51c498f-c126-4f4e-a459-cff689ecf93d None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:21.780058 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb34bcd1-c48d-4357-95fd-e797fd8879f2 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:21.805108 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0b883f-735a-4b45-811c-ecbdac0fba1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:21.805723 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0b883f-735a-4b45-811c-ecbdac0fba1c None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:21.806004 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0b883f-735a-4b45-811c-ecbdac0fba1c None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:21.806279 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0b883f-735a-4b45-811c-ecbdac0fba1c None admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:21.806976 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0b883f-735a-4b45-811c-ecbdac0fba1c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:21.807284 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0b883f-735a-4b45-811c-ecbdac0fba1c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:21.807872 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:21.809234 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0b883f-735a-4b45-811c-ecbdac0fba1c None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:21.822675 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 264/537] 10.222.0.16 () {56 vars in 1148 bytes} [Wed Aug 24 09:08:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:21.824369 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0b883f-735a-4b45-811c-ecbdac0fba1c None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:21.826004 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb4ab64-87fa-43cc-a2a9-3edb13b98ea2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:21.826876 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb4ab64-87fa-43cc-a2a9-3edb13b98ea2 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:21.827856 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb4ab64-87fa-43cc-a2a9-3edb13b98ea2 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:21.829500 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb4ab64-87fa-43cc-a2a9-3edb13b98ea2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:21.830464 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb4ab64-87fa-43cc-a2a9-3edb13b98ea2 None admin] RBAC: Adding query filter params (role.id=67fd70a7d9794f7fb8faebffdec46dd2, scope.project.id=c467e88e2fb0491bacefcf6d2ce326d9, user.id=ea2666d430a646d386104addf1f42888) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:21.830979 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb4ab64-87fa-43cc-a2a9-3edb13b98ea2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:21.832653 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb4ab64-87fa-43cc-a2a9-3edb13b98ea2 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:21.841911 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 274/538] 10.222.0.16 () {56 vars in 1374 bytes} [Wed Aug 24 09:08:21 2022] GET /identity/v3/role_assignments?user.id=ea2666d430a646d386104addf1f42888&scope.project.id=c467e88e2fb0491bacefcf6d2ce326d9&role.id=67fd70a7d9794f7fb8faebffdec46dd2 => generated 259 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:21.843802 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb4ab64-87fa-43cc-a2a9-3edb13b98ea2 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:21.941074 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0b883f-735a-4b45-811c-ecbdac0fba1c None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:21.975487 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb4ab64-87fa-43cc-a2a9-3edb13b98ea2 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:21.977703 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623ea2d8-424b-4b25-9efe-ae976952ffe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:21.978319 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623ea2d8-424b-4b25-9efe-ae976952ffe3 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:21.978604 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623ea2d8-424b-4b25-9efe-ae976952ffe3 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:21.978881 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623ea2d8-424b-4b25-9efe-ae976952ffe3 None admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:21.981003 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623ea2d8-424b-4b25-9efe-ae976952ffe3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:21.981003 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623ea2d8-424b-4b25-9efe-ae976952ffe3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:21.981943 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623ea2d8-424b-4b25-9efe-ae976952ffe3 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:21.995016 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 265/539] 10.222.0.16 () {56 vars in 1148 bytes} [Wed Aug 24 09:08:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:22.005169 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623ea2d8-424b-4b25-9efe-ae976952ffe3 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:22.018647 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f419f706-1488-4734-b4a8-981d53b8c0e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:22.019697 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f419f706-1488-4734-b4a8-981d53b8c0e9 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:22.020245 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f419f706-1488-4734-b4a8-981d53b8c0e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:22.020771 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f419f706-1488-4734-b4a8-981d53b8c0e9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:22.021810 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f419f706-1488-4734-b4a8-981d53b8c0e9 None admin] RBAC: Adding query filter params (role.id=67fd70a7d9794f7fb8faebffdec46dd2, scope.project.id=c467e88e2fb0491bacefcf6d2ce326d9, user.id=8e5260e798404e8bb12b25ca8ea97052) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:22.022429 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f419f706-1488-4734-b4a8-981d53b8c0e9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:22.024290 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f419f706-1488-4734-b4a8-981d53b8c0e9 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:22.032376 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 275/540] 10.222.0.16 () {56 vars in 1374 bytes} [Wed Aug 24 09:08:21 2022] GET /identity/v3/role_assignments?user.id=8e5260e798404e8bb12b25ca8ea97052&scope.project.id=c467e88e2fb0491bacefcf6d2ce326d9&role.id=67fd70a7d9794f7fb8faebffdec46dd2 => generated 259 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:22.042354 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:22.140101 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623ea2d8-424b-4b25-9efe-ae976952ffe3 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:22.179399 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab91e2a-e2a3-4d04-b22c-18f60071fa34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:22.180406 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab91e2a-e2a3-4d04-b22c-18f60071fa34 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:22.180894 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab91e2a-e2a3-4d04-b22c-18f60071fa34 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:22.181547 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab91e2a-e2a3-4d04-b22c-18f60071fa34 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:22.182508 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab91e2a-e2a3-4d04-b22c-18f60071fa34 None admin] RBAC: Adding query filter params (role.id=67fd70a7d9794f7fb8faebffdec46dd2, scope.project.id=c467e88e2fb0491bacefcf6d2ce326d9, user.id=05c32b83577c46488906fe409a790b24) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:22.183101 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab91e2a-e2a3-4d04-b22c-18f60071fa34 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:22.184811 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab91e2a-e2a3-4d04-b22c-18f60071fa34 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:22.191696 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 266/541] 10.222.0.16 () {56 vars in 1374 bytes} [Wed Aug 24 09:08:22 2022] GET /identity/v3/role_assignments?user.id=05c32b83577c46488906fe409a790b24&scope.project.id=c467e88e2fb0491bacefcf6d2ce326d9&role.id=67fd70a7d9794f7fb8faebffdec46dd2 => generated 259 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:22.379235 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1709e1-0337-403f-8164-970b8bfc0c54 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:22.379764 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1709e1-0337-403f-8164-970b8bfc0c54 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:22.379838 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1709e1-0337-403f-8164-970b8bfc0c54 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:22.381291 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 276/542] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:08:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:08:22.387135 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850829fe-f695-4085-8edb-d796715624d3 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:22.387420 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850829fe-f695-4085-8edb-d796715624d3 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:22.387729 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850829fe-f695-4085-8edb-d796715624d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:22.409353 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-850829fe-f695-4085-8edb-d796715624d3 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:08:22.410573 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-850829fe-f695-4085-8edb-d796715624d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:22.497078 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 267/543] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:08:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:08:22.604651 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9baa2df-4aa4-4615-a9a9-3c9eae3d2391 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:22.605479 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9baa2df-4aa4-4615-a9a9-3c9eae3d2391 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:22.605721 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9baa2df-4aa4-4615-a9a9-3c9eae3d2391 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:22.607325 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 277/544] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:08:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:08:22.612804 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f52fd9-6463-4c1a-a7f8-bfb53c718328 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:22.613101 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f52fd9-6463-4c1a-a7f8-bfb53c718328 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:22.613382 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f52fd9-6463-4c1a-a7f8-bfb53c718328 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:22.639777 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-e0f52fd9-6463-4c1a-a7f8-bfb53c718328 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:08:22.641001 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-e0f52fd9-6463-4c1a-a7f8-bfb53c718328 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:22.645460 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:22.733909 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 268/545] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:08:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 09:08:22.940364 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9baa2df-4aa4-4615-a9a9-3c9eae3d2391 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:23.005871 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:23.065222 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9baa2df-4aa4-4615-a9a9-3c9eae3d2391 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:23.110928 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16eb29b8-8f64-4536-a8a0-c279e9bb4a26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:23.111567 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16eb29b8-8f64-4536-a8a0-c279e9bb4a26 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:23.111869 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16eb29b8-8f64-4536-a8a0-c279e9bb4a26 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:23.112147 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16eb29b8-8f64-4536-a8a0-c279e9bb4a26 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:23.122808 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16eb29b8-8f64-4536-a8a0-c279e9bb4a26 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:08:23.132004 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-16eb29b8-8f64-4536-a8a0-c279e9bb4a26 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 09:08:23.132004 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:08:23.132004 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:08:23.132004 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:08:23.132004 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:08:23.132004 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:08:23.132004 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:08:23.132004 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:08:23.132004 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:08:23.132004 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:08:23.132004 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:08:23.132004 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:08:23.132004 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:08:23.132004 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:08:23.132004 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:08:23.132004 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:08:23.132898 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:08:23.132898 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:08:23.132898 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:08:23.132898 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:08:23.132898 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:08:23.132898 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:08:23.132898 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:08:23.132898 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:08:23.132898 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:08:23.132898 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:08:23.132898 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:08:23.132898 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 24 09:08:23.132898 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:08:23.132898 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:08:23.132898 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:08:23.132898 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:08:23.133693 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:08:23.133693 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:08:23.133693 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:08:23.133693 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:08:23.133693 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:08:23.133693 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:08:23.133693 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:08:23.133693 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:08:23.133693 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:08:23.133693 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:08:23.133693 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 09:08:23.133693 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 24 09:08:23.133693 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 278/546] 10.222.0.16 () {56 vars in 1102 bytes} [Wed Aug 24 09:08:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:08:23.139711 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f52fd9-6463-4c1a-a7f8-bfb53c718328 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:23.150257 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16eb29b8-8f64-4536-a8a0-c279e9bb4a26 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:23.268477 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f52fd9-6463-4c1a-a7f8-bfb53c718328 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:23.332735 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e422031-8bf1-434d-b085-3de15e6228c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:23.333366 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e422031-8bf1-434d-b085-3de15e6228c7 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:23.333648 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e422031-8bf1-434d-b085-3de15e6228c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:23.333924 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e422031-8bf1-434d-b085-3de15e6228c7 None admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:23.334605 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e422031-8bf1-434d-b085-3de15e6228c7 None admin] RBAC: Adding query filter params (name=service) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:23.334916 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e422031-8bf1-434d-b085-3de15e6228c7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:23.336353 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e422031-8bf1-434d-b085-3de15e6228c7 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:23.352968 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16eb29b8-8f64-4536-a8a0-c279e9bb4a26 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:23.356867 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 269/547] 10.222.0.16 () {56 vars in 1103 bytes} [Wed Aug 24 09:08:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:23.362489 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e422031-8bf1-434d-b085-3de15e6228c7 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:23.368657 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:23.369180 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:23.418788 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6412a03b-f8a7-478b-9a58-f49d918198e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:23.419410 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6412a03b-f8a7-478b-9a58-f49d918198e3 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:23.419718 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6412a03b-f8a7-478b-9a58-f49d918198e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:23.420004 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6412a03b-f8a7-478b-9a58-f49d918198e3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:23.442571 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6412a03b-f8a7-478b-9a58-f49d918198e3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:08:23.447895 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-6412a03b-f8a7-478b-9a58-f49d918198e3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 09:08:23.447895 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:08:23.447895 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:08:23.447895 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:08:23.447895 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:08:23.447895 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:08:23.447895 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:08:23.447895 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:08:23.447895 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:08:23.447895 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:08:23.447895 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:08:23.447895 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:08:23.447895 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:08:23.447895 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:08:23.447895 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:08:23.447895 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:08:23.448874 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:08:23.448874 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:08:23.448874 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:08:23.448874 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:08:23.448874 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:08:23.448874 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:08:23.448874 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:08:23.448874 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:08:23.448874 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:08:23.448874 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:08:23.448874 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:08:23.448874 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 24 09:08:23.448874 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:08:23.448874 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:08:23.448874 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:08:23.448874 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:08:23.449762 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:08:23.449762 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:08:23.449762 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:08:23.449762 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:08:23.449762 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:08:23.449762 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:08:23.449762 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:08:23.449762 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:08:23.449762 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:08:23.449762 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:08:23.449762 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 09:08:23.449762 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 24 09:08:23.449762 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 279/548] 10.222.0.16 () {56 vars in 1102 bytes} [Wed Aug 24 09:08:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 09:08:23.457428 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6412a03b-f8a7-478b-9a58-f49d918198e3 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:23.569755 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e422031-8bf1-434d-b085-3de15e6228c7 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:23.600483 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6412a03b-f8a7-478b-9a58-f49d918198e3 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:23.640351 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ba8780-6473-4f49-854b-1416541eacf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:23.641734 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ba8780-6473-4f49-854b-1416541eacf8 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:23.642705 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ba8780-6473-4f49-854b-1416541eacf8 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:23.643608 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ba8780-6473-4f49-854b-1416541eacf8 None admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:23.644779 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ba8780-6473-4f49-854b-1416541eacf8 None admin] RBAC: Adding query filter params (name=service) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:23.645775 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ba8780-6473-4f49-854b-1416541eacf8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:23.646560 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29123a65-2f5d-46f6-bba5-0d80a21b6c82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:23.647836 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29123a65-2f5d-46f6-bba5-0d80a21b6c82 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:23.648146 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29123a65-2f5d-46f6-bba5-0d80a21b6c82 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:23.648432 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29123a65-2f5d-46f6-bba5-0d80a21b6c82 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:23.649507 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ba8780-6473-4f49-854b-1416541eacf8 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:23.659046 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 280/549] 10.222.0.16 () {56 vars in 1103 bytes} [Wed Aug 24 09:08:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:08:23.664750 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ba8780-6473-4f49-854b-1416541eacf8 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:23.673774 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29123a65-2f5d-46f6-bba5-0d80a21b6c82 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:23.675267 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29123a65-2f5d-46f6-bba5-0d80a21b6c82 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:23.699964 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-29123a65-2f5d-46f6-bba5-0d80a21b6c82 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:08:23.699964 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:08:23.699964 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:08:23.699964 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:08:23.699964 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:08:23.699964 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:08:23.699964 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:08:23.699964 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:08:23.699964 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:08:23.699964 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:08:23.699964 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:08:23.699964 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:08:23.699964 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:08:23.699964 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:08:23.699964 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:08:23.699964 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:08:23.700932 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:08:23.700932 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:08:23.700932 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:08:23.700932 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:08:23.700932 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:08:23.700932 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:08:23.700932 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:08:23.700932 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Aug 24 09:08:23.700932 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:08:23.700932 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:08:23.700932 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:08:23.700932 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:08:23.700932 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:08:23.700932 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:08:23.700932 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:08:23.700932 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:08:23.700932 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:08:23.701814 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:08:23.701814 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:08:23.701814 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Aug 24 09:08:23.704574 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 270/550] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 24 09:08:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 342 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 09:08:23.710349 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29123a65-2f5d-46f6-bba5-0d80a21b6c82 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:23.710894 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:23.799744 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:23.839040 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ba8780-6473-4f49-854b-1416541eacf8 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:23.871531 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29123a65-2f5d-46f6-bba5-0d80a21b6c82 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:23.911974 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce6d0f7-f870-48c1-be57-e99ea74c6d87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:23.912970 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce6d0f7-f870-48c1-be57-e99ea74c6d87 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:23.913599 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce6d0f7-f870-48c1-be57-e99ea74c6d87 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:23.915117 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce6d0f7-f870-48c1-be57-e99ea74c6d87 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:23.935968 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce6d0f7-f870-48c1-be57-e99ea74c6d87 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:23.937746 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce6d0f7-f870-48c1-be57-e99ea74c6d87 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:23.949023 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9e69f0-3b2d-4599-8916-70ca0716d71e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:23.949887 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9e69f0-3b2d-4599-8916-70ca0716d71e None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:23.950185 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9e69f0-3b2d-4599-8916-70ca0716d71e None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:23.950455 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9e69f0-3b2d-4599-8916-70ca0716d71e None admin] PATH_INFO: `/v3/domains` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:23.951107 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9e69f0-3b2d-4599-8916-70ca0716d71e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:23.951410 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9e69f0-3b2d-4599-8916-70ca0716d71e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:23.952815 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9e69f0-3b2d-4599-8916-70ca0716d71e None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:23.960347 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-5ce6d0f7-f870-48c1-be57-e99ea74c6d87 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:08:23.960347 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:08:23.960347 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:08:23.960347 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:08:23.960347 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:08:23.960347 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:08:23.960347 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:08:23.960347 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:08:23.960347 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:08:23.960347 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:08:23.960347 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:08:23.960347 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:08:23.960347 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:08:23.960347 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:08:23.960347 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:08:23.960347 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:08:23.969928 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:08:23.969928 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:08:23.969928 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:08:23.969928 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:08:23.969928 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:08:23.969928 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:08:23.969928 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:08:23.969928 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 24 09:08:23.969928 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:08:23.969928 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:08:23.969928 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:08:23.969928 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:08:23.969928 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:08:23.969928 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:08:23.969928 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:08:23.969928 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:08:23.969928 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:08:23.972083 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:08:23.972083 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:08:23.972083 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 24 09:08:23.972460 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 271/551] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Aug 24 09:08:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:23.972612 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 281/552] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 24 09:08:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 09:08:23.974697 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9e69f0-3b2d-4599-8916-70ca0716d71e None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:23.975778 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce6d0f7-f870-48c1-be57-e99ea74c6d87 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:24.145392 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9e69f0-3b2d-4599-8916-70ca0716d71e None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:24.159944 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:24.173643 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:24.196374 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce6d0f7-f870-48c1-be57-e99ea74c6d87 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:24.214855 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae01062c-408a-4c18-9ca1-f0fc0399f804 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:24.215553 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae01062c-408a-4c18-9ca1-f0fc0399f804 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:24.215850 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae01062c-408a-4c18-9ca1-f0fc0399f804 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:24.216123 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae01062c-408a-4c18-9ca1-f0fc0399f804 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:24.226883 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae01062c-408a-4c18-9ca1-f0fc0399f804 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:24.228370 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae01062c-408a-4c18-9ca1-f0fc0399f804 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:24.254805 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-ae01062c-408a-4c18-9ca1-f0fc0399f804 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 09:08:24.254805 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:08:24.254805 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:08:24.254805 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:08:24.254805 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:08:24.254805 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:08:24.254805 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:08:24.254805 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:08:24.254805 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:08:24.254805 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:08:24.254805 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:08:24.254805 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:08:24.254805 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 09:08:24.254805 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 09:08:24.254805 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 09:08:24.254805 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 09:08:24.256064 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:08:24.256064 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:08:24.256064 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 09:08:24.256064 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:08:24.256064 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 09:08:24.256064 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:08:24.256064 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:08:24.256064 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:08:24.256064 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:08:24.256064 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:08:24.256064 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:08:24.256064 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Aug 24 09:08:24.256064 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:08:24.256064 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:08:24.256064 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:08:24.256064 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:08:24.256064 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:08:24.256064 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:08:24.257372 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:08:24.257372 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:08:24.257372 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 09:08:24.257372 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 09:08:24.257372 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 09:08:24.257372 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 09:08:24.257372 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 09:08:24.257372 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 09:08:24.257372 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 09:08:24.257372 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Aug 24 09:08:24.257372 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 272/553] 10.222.0.16 () {56 vars in 1093 bytes} [Wed Aug 24 09:08:23 2022] GET /identity/v3/users/nova => generated 82 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:08:24.263701 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae01062c-408a-4c18-9ca1-f0fc0399f804 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:24.269140 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f736880-10e8-41ad-b6f2-104f538a13fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:24.269947 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f736880-10e8-41ad-b6f2-104f538a13fd None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:24.270380 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f736880-10e8-41ad-b6f2-104f538a13fd None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:24.270804 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f736880-10e8-41ad-b6f2-104f538a13fd None admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:24.271630 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f736880-10e8-41ad-b6f2-104f538a13fd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:24.272120 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f736880-10e8-41ad-b6f2-104f538a13fd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:24.273635 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f736880-10e8-41ad-b6f2-104f538a13fd None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:24.304377 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 282/554] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Aug 24 09:08:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:24.308651 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd0257d-5560-439d-a7d3-c342cd91484d None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:24.308950 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd0257d-5560-439d-a7d3-c342cd91484d None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:24.309232 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd0257d-5560-439d-a7d3-c342cd91484d None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:24.310831 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 283/555] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:08:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 09:08:24.313144 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd0257d-5560-439d-a7d3-c342cd91484d None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:24.386517 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae01062c-408a-4c18-9ca1-f0fc0399f804 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:24.421504 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd0257d-5560-439d-a7d3-c342cd91484d None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:24.422860 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2157e56-3f3a-46a3-b0a8-35c7a72e9317 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:24.423609 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2157e56-3f3a-46a3-b0a8-35c7a72e9317 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:24.423905 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2157e56-3f3a-46a3-b0a8-35c7a72e9317 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:24.424208 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2157e56-3f3a-46a3-b0a8-35c7a72e9317 None admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:24.425030 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2157e56-3f3a-46a3-b0a8-35c7a72e9317 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:24.425343 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2157e56-3f3a-46a3-b0a8-35c7a72e9317 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:24.426776 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2157e56-3f3a-46a3-b0a8-35c7a72e9317 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:24.452150 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 273/556] 10.222.0.16 () {56 vars in 1133 bytes} [Wed Aug 24 09:08:24 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:24.453400 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5eca3f6-e320-4a51-a1fe-d1a923dd4d0b None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:24.453696 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5eca3f6-e320-4a51-a1fe-d1a923dd4d0b None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:24.453986 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5eca3f6-e320-4a51-a1fe-d1a923dd4d0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:24.458424 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d0de82-64cc-4829-9fee-c26fbe0bc89b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:24.459060 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d0de82-64cc-4829-9fee-c26fbe0bc89b None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:24.459342 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d0de82-64cc-4829-9fee-c26fbe0bc89b None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:24.459647 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d0de82-64cc-4829-9fee-c26fbe0bc89b None admin] PATH_INFO: `/v3/users/swift` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:24.468579 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d0de82-64cc-4829-9fee-c26fbe0bc89b None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:24.470015 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d0de82-64cc-4829-9fee-c26fbe0bc89b None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:24.472658 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-b5eca3f6-e320-4a51-a1fe-d1a923dd4d0b None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:08:24.473872 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-b5eca3f6-e320-4a51-a1fe-d1a923dd4d0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:24.481938 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-21d0de82-64cc-4829-9fee-c26fbe0bc89b None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 09:08:24.481938 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:08:24.481938 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:08:24.481938 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:08:24.481938 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:08:24.481938 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:08:24.481938 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:08:24.481938 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:08:24.481938 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:08:24.481938 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:08:24.481938 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:08:24.481938 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:08:24.481938 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 09:08:24.481938 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 09:08:24.481938 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 09:08:24.481938 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 09:08:24.488207 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:08:24.488207 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:08:24.488207 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 09:08:24.488207 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:08:24.488207 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 09:08:24.488207 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:08:24.488207 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:08:24.488207 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:08:24.488207 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:08:24.488207 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:08:24.488207 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:08:24.488207 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 24 09:08:24.488207 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:08:24.488207 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:08:24.488207 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:08:24.488207 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:08:24.488207 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:08:24.488207 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:08:24.489174 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:08:24.489174 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:08:24.489174 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 09:08:24.489174 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 09:08:24.489174 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 09:08:24.489174 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 09:08:24.489174 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 09:08:24.489174 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 09:08:24.489174 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 09:08:24.489174 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 24 09:08:24.489174 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 284/557] 10.222.0.16 () {56 vars in 1096 bytes} [Wed Aug 24 09:08:24 2022] GET /identity/v3/users/swift => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:08:24.489174 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6173fb59-9cd3-4231-9a41-e5ef174c7840 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:24.489174 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6173fb59-9cd3-4231-9a41-e5ef174c7840 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:24.489892 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6173fb59-9cd3-4231-9a41-e5ef174c7840 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:24.489892 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 285/558] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:08:24 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:08:24.489892 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6173fb59-9cd3-4231-9a41-e5ef174c7840 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:24.490296 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:24.501683 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:24.541597 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 274/559] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:08:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:08:24.543488 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac89e00-6ba4-494f-ba61-4245034949ba None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:24.543866 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac89e00-6ba4-494f-ba61-4245034949ba None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:24.544155 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac89e00-6ba4-494f-ba61-4245034949ba None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:24.545509 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 275/560] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:08:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:08:24.546147 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac89e00-6ba4-494f-ba61-4245034949ba None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:24.598001 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6173fb59-9cd3-4231-9a41-e5ef174c7840 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:24.633046 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1ae35d-0306-4bd5-8609-22db7567fb2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:24.633677 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1ae35d-0306-4bd5-8609-22db7567fb2d None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:24.633958 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1ae35d-0306-4bd5-8609-22db7567fb2d None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:24.634232 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1ae35d-0306-4bd5-8609-22db7567fb2d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:24.645568 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1ae35d-0306-4bd5-8609-22db7567fb2d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:24.647024 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1ae35d-0306-4bd5-8609-22db7567fb2d None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:24.662314 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-5e1ae35d-0306-4bd5-8609-22db7567fb2d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:08:24.662314 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:08:24.662314 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:08:24.662314 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:08:24.662314 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:08:24.662314 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:08:24.662314 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:08:24.662314 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:08:24.662314 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:08:24.662314 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:08:24.662314 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:08:24.662314 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:08:24.662314 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:08:24.662314 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:08:24.662314 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:08:24.662314 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:08:24.663492 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:08:24.663492 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:08:24.663492 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:08:24.663492 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:08:24.663492 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:08:24.663492 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:08:24.663492 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:08:24.663492 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 24 09:08:24.663492 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:08:24.663492 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:08:24.663492 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:08:24.663492 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:08:24.663492 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:08:24.663492 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:08:24.663492 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:08:24.663492 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:08:24.663492 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:08:24.664587 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:08:24.664587 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:08:24.664587 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 24 09:08:24.664794 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac89e00-6ba4-494f-ba61-4245034949ba None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:24.666375 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 286/561] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 24 09:08:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:08:24.667790 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00818f6-0bf9-40bd-98a4-8bd2c7df9043 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:24.668236 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00818f6-0bf9-40bd-98a4-8bd2c7df9043 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:24.668671 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00818f6-0bf9-40bd-98a4-8bd2c7df9043 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:24.687460 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-a00818f6-0bf9-40bd-98a4-8bd2c7df9043 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:08:24.688617 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-a00818f6-0bf9-40bd-98a4-8bd2c7df9043 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:24.700578 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c8e987-0fa6-4b92-93b6-1fe60c1ed0af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:24.701190 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c8e987-0fa6-4b92-93b6-1fe60c1ed0af None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:24.701474 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c8e987-0fa6-4b92-93b6-1fe60c1ed0af None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:24.701751 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c8e987-0fa6-4b92-93b6-1fe60c1ed0af None admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:24.702501 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c8e987-0fa6-4b92-93b6-1fe60c1ed0af None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:24.702788 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c8e987-0fa6-4b92-93b6-1fe60c1ed0af None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:24.704500 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c8e987-0fa6-4b92-93b6-1fe60c1ed0af None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:24.727670 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 276/562] 10.222.0.16 () {56 vars in 1135 bytes} [Wed Aug 24 09:08:24 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:24.727670 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e013374e-397a-4ad6-8e05-b41fad8d0af2 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:24.727670 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e013374e-397a-4ad6-8e05-b41fad8d0af2 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:24.727670 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e013374e-397a-4ad6-8e05-b41fad8d0af2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:24.743552 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-e013374e-397a-4ad6-8e05-b41fad8d0af2 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:08:24.744780 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-e013374e-397a-4ad6-8e05-b41fad8d0af2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:24.748769 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:24.759469 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:24.767445 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 287/563] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:08:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:08:24.768449 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00818f6-0bf9-40bd-98a4-8bd2c7df9043 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:24.819967 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 277/564] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:08:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:08:24.821026 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e013374e-397a-4ad6-8e05-b41fad8d0af2 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:24.897362 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00818f6-0bf9-40bd-98a4-8bd2c7df9043 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:24.938348 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e013374e-397a-4ad6-8e05-b41fad8d0af2 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:24.949164 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf37196-0556-4f3c-8066-c7cfa528b30a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:24.949789 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf37196-0556-4f3c-8066-c7cfa528b30a None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:24.950074 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf37196-0556-4f3c-8066-c7cfa528b30a None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:24.950353 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf37196-0556-4f3c-8066-c7cfa528b30a None admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:24.951028 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf37196-0556-4f3c-8066-c7cfa528b30a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:24.951780 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf37196-0556-4f3c-8066-c7cfa528b30a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:24.953477 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf37196-0556-4f3c-8066-c7cfa528b30a None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:24.971365 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 288/565] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Aug 24 09:08:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:24.972503 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf37196-0556-4f3c-8066-c7cfa528b30a None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:24.984615 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2215afb7-c1a9-4bb1-a48a-00cae578ec74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:24.985231 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2215afb7-c1a9-4bb1-a48a-00cae578ec74 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:24.985516 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2215afb7-c1a9-4bb1-a48a-00cae578ec74 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:24.985794 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2215afb7-c1a9-4bb1-a48a-00cae578ec74 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:25.014082 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:25.023805 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:25.070527 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2215afb7-c1a9-4bb1-a48a-00cae578ec74 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:25.072102 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2215afb7-c1a9-4bb1-a48a-00cae578ec74 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:25.082198 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf37196-0556-4f3c-8066-c7cfa528b30a None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:25.117518 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e84e1c6-5a4b-4b13-92ba-d266d9b73a2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:25.118149 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e84e1c6-5a4b-4b13-92ba-d266d9b73a2c None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:25.118427 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e84e1c6-5a4b-4b13-92ba-d266d9b73a2c None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:25.118697 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e84e1c6-5a4b-4b13-92ba-d266d9b73a2c None admin] PATH_INFO: `/v3/roles/service` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:25.129292 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e84e1c6-5a4b-4b13-92ba-d266d9b73a2c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:08:25.134448 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-0e84e1c6-5a4b-4b13-92ba-d266d9b73a2c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 09:08:25.134448 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:08:25.134448 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:08:25.134448 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:08:25.134448 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:08:25.134448 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:08:25.134448 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:08:25.134448 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:08:25.134448 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:08:25.134448 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:08:25.134448 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:08:25.134448 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:08:25.134448 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:08:25.134448 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:08:25.134448 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:08:25.134448 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:08:25.135386 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:08:25.135386 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:08:25.135386 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:08:25.135386 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:08:25.135386 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:08:25.135386 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:08:25.135386 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:08:25.135386 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:08:25.135386 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:08:25.135386 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:08:25.135386 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:08:25.135386 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 24 09:08:25.135386 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:08:25.135386 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:08:25.135386 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:08:25.135386 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:08:25.136912 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:08:25.136912 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:08:25.136912 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:08:25.136912 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:08:25.136912 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:08:25.136912 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:08:25.136912 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:08:25.136912 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:08:25.136912 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:08:25.136912 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:08:25.136912 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 09:08:25.136912 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 24 09:08:25.136912 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 289/566] 10.222.0.16 () {56 vars in 1102 bytes} [Wed Aug 24 09:08:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:08:25.137634 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e84e1c6-5a4b-4b13-92ba-d266d9b73a2c None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:25.174292 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 278/567] 10.222.0.16 () {58 vars in 1282 bytes} [Wed Aug 24 09:08:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 09:08:25.175242 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2215afb7-c1a9-4bb1-a48a-00cae578ec74 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:25.237480 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:25.256591 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e84e1c6-5a4b-4b13-92ba-d266d9b73a2c None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:25.258854 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:25.286001 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2215afb7-c1a9-4bb1-a48a-00cae578ec74 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:25.289880 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65865bf0-8634-48fd-b088-88b08697b66e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:25.290511 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65865bf0-8634-48fd-b088-88b08697b66e None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:25.290785 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65865bf0-8634-48fd-b088-88b08697b66e None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:25.291058 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65865bf0-8634-48fd-b088-88b08697b66e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:25.297546 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65865bf0-8634-48fd-b088-88b08697b66e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:25.298997 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65865bf0-8634-48fd-b088-88b08697b66e None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:25.310562 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-65865bf0-8634-48fd-b088-88b08697b66e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:08:25.310562 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:08:25.310562 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:08:25.310562 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:08:25.310562 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:08:25.310562 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:08:25.310562 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:08:25.310562 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:08:25.310562 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:08:25.310562 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:08:25.310562 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:08:25.310562 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:08:25.310562 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 09:08:25.310562 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 09:08:25.310562 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 09:08:25.311427 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 09:08:25.311427 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:08:25.311427 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:08:25.311427 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:08:25.311427 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:08:25.311427 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:08:25.311427 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:08:25.311427 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:08:25.311427 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 24 09:08:25.311427 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:08:25.311427 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:08:25.311427 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:08:25.311427 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:08:25.311427 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:08:25.311427 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:08:25.311427 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:08:25.311427 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:08:25.312345 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 09:08:25.312345 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 09:08:25.312345 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 09:08:25.312345 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 09:08:25.312345 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 09:08:25.312345 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 09:08:25.312345 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:08:25.312345 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 24 09:08:25.312345 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 290/568] 10.222.0.16 () {56 vars in 1111 bytes} [Wed Aug 24 09:08:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:08:25.312824 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65865bf0-8634-48fd-b088-88b08697b66e None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:25.326443 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff1069f-c9f6-4d80-93f0-2fd5ec9265b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:25.327078 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff1069f-c9f6-4d80-93f0-2fd5ec9265b2 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:25.327357 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff1069f-c9f6-4d80-93f0-2fd5ec9265b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:25.327657 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff1069f-c9f6-4d80-93f0-2fd5ec9265b2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:25.338015 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff1069f-c9f6-4d80-93f0-2fd5ec9265b2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:08:25.342779 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-eff1069f-c9f6-4d80-93f0-2fd5ec9265b2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 09:08:25.342779 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:08:25.342779 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:08:25.342779 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:08:25.342779 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:08:25.342779 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:08:25.342779 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:08:25.342779 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:08:25.342779 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:08:25.342779 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:08:25.342779 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:08:25.342779 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:08:25.342779 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:08:25.342779 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:08:25.342779 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:08:25.342779 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:08:25.343820 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:08:25.343820 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:08:25.343820 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:08:25.343820 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:08:25.343820 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:08:25.343820 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:08:25.343820 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:08:25.343820 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:08:25.343820 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:08:25.343820 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:08:25.343820 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:08:25.343820 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Aug 24 09:08:25.343820 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:08:25.343820 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:08:25.343820 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:08:25.343820 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:08:25.344681 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:08:25.344681 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:08:25.344681 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:08:25.344681 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:08:25.344681 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:08:25.344681 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:08:25.344681 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:08:25.344681 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:08:25.344681 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:08:25.344681 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:08:25.344681 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 09:08:25.344681 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Aug 24 09:08:25.344681 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 279/569] 10.222.0.16 () {56 vars in 1102 bytes} [Wed Aug 24 09:08:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:08:25.344681 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff1069f-c9f6-4d80-93f0-2fd5ec9265b2 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:25.414457 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65865bf0-8634-48fd-b088-88b08697b66e None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:25.442759 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff1069f-c9f6-4d80-93f0-2fd5ec9265b2 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:25.449588 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418a1058-d5bb-4fc7-840f-30f0295957a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:25.450610 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418a1058-d5bb-4fc7-840f-30f0295957a9 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:25.450893 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418a1058-d5bb-4fc7-840f-30f0295957a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:25.451168 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418a1058-d5bb-4fc7-840f-30f0295957a9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:25.453138 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:25.461800 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418a1058-d5bb-4fc7-840f-30f0295957a9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:08:25.466803 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-418a1058-d5bb-4fc7-840f-30f0295957a9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 09:08:25.466803 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:08:25.466803 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:08:25.466803 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:08:25.466803 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:08:25.466803 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:08:25.466803 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:08:25.466803 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:08:25.466803 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:08:25.466803 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:08:25.466803 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:08:25.466803 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:08:25.466803 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:08:25.466803 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:08:25.466803 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:08:25.466803 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:08:25.467902 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:08:25.467902 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:08:25.467902 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:08:25.467902 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:08:25.467902 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:08:25.467902 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:08:25.467902 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:08:25.467902 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:08:25.467902 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:08:25.467902 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:08:25.467902 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:08:25.467902 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 24 09:08:25.467902 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:08:25.467902 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:08:25.467902 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:08:25.467902 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:08:25.468749 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:08:25.468749 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:08:25.468749 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:08:25.468749 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:08:25.468749 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:08:25.468749 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:08:25.468749 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:08:25.468749 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:08:25.468749 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:08:25.468749 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:08:25.468749 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 09:08:25.468749 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 24 09:08:25.468749 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 291/570] 10.222.0.16 () {56 vars in 1102 bytes} [Wed Aug 24 09:08:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:08:25.468749 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418a1058-d5bb-4fc7-840f-30f0295957a9 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:25.478603 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6f9ba1-9c73-48c0-9232-771431da1ba5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:25.479210 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6f9ba1-9c73-48c0-9232-771431da1ba5 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:25.479490 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6f9ba1-9c73-48c0-9232-771431da1ba5 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:25.480331 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6f9ba1-9c73-48c0-9232-771431da1ba5 None admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:25.480998 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6f9ba1-9c73-48c0-9232-771431da1ba5 None admin] RBAC: Adding query filter params (name=service) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:25.481302 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6f9ba1-9c73-48c0-9232-771431da1ba5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:25.482681 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6f9ba1-9c73-48c0-9232-771431da1ba5 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:25.491126 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 280/571] 10.222.0.16 () {56 vars in 1103 bytes} [Wed Aug 24 09:08:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:25.491953 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6f9ba1-9c73-48c0-9232-771431da1ba5 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:25.499927 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:25.573524 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418a1058-d5bb-4fc7-840f-30f0295957a9 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:25.590136 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6f9ba1-9c73-48c0-9232-771431da1ba5 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:25.607059 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd6f1e0-de42-490c-be7a-2f5d23cbdc6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:25.607720 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd6f1e0-de42-490c-be7a-2f5d23cbdc6d None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:25.608004 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd6f1e0-de42-490c-be7a-2f5d23cbdc6d None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:25.608279 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd6f1e0-de42-490c-be7a-2f5d23cbdc6d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:25.618222 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd6f1e0-de42-490c-be7a-2f5d23cbdc6d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:25.619934 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd6f1e0-de42-490c-be7a-2f5d23cbdc6d None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:25.623394 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f0a492-29da-4738-91ba-8494528192ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:25.624013 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f0a492-29da-4738-91ba-8494528192ec None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:25.624293 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f0a492-29da-4738-91ba-8494528192ec None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:25.624568 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f0a492-29da-4738-91ba-8494528192ec None admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:25.625251 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f0a492-29da-4738-91ba-8494528192ec None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:25.625556 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f0a492-29da-4738-91ba-8494528192ec None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:25.626969 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f0a492-29da-4738-91ba-8494528192ec None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:25.634659 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-5cd6f1e0-de42-490c-be7a-2f5d23cbdc6d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:08:25.634659 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:08:25.634659 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:08:25.634659 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:08:25.634659 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:08:25.634659 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:08:25.634659 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:08:25.634659 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:08:25.634659 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:08:25.634659 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:08:25.634659 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:08:25.634659 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:08:25.634659 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:08:25.634659 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:08:25.634659 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:08:25.634659 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:08:25.635727 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:08:25.635727 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:08:25.635727 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:08:25.635727 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:08:25.635727 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:08:25.635727 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:08:25.635727 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:08:25.635727 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 24 09:08:25.635727 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:08:25.635727 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:08:25.635727 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:08:25.635727 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:08:25.635727 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:08:25.635727 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:08:25.635727 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:08:25.635727 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:08:25.635727 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:08:25.636860 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:08:25.636860 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:08:25.636860 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 24 09:08:25.636860 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 292/572] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 24 09:08:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:08:25.636860 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd6f1e0-de42-490c-be7a-2f5d23cbdc6d None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:25.642141 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 281/573] 10.222.0.16 () {56 vars in 1148 bytes} [Wed Aug 24 09:08:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:25.643028 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f0a492-29da-4738-91ba-8494528192ec None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:25.675148 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:25.713302 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:25.735372 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd6f1e0-de42-490c-be7a-2f5d23cbdc6d None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:25.742186 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f0a492-29da-4738-91ba-8494528192ec None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:25.768799 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853935de-1cb3-4f4d-a0c8-3c84573053e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:25.769396 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853935de-1cb3-4f4d-a0c8-3c84573053e6 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:25.769668 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853935de-1cb3-4f4d-a0c8-3c84573053e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:25.769939 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853935de-1cb3-4f4d-a0c8-3c84573053e6 None admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:25.770585 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853935de-1cb3-4f4d-a0c8-3c84573053e6 None admin] RBAC: Adding query filter params (name=service) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:25.770882 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853935de-1cb3-4f4d-a0c8-3c84573053e6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:25.772248 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853935de-1cb3-4f4d-a0c8-3c84573053e6 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:25.776602 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d72a7ea-4e49-42c2-93ca-8f6beb8e987d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:25.777702 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d72a7ea-4e49-42c2-93ca-8f6beb8e987d None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:25.777981 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d72a7ea-4e49-42c2-93ca-8f6beb8e987d None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:25.778250 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d72a7ea-4e49-42c2-93ca-8f6beb8e987d None admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:25.778902 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d72a7ea-4e49-42c2-93ca-8f6beb8e987d None admin] RBAC: Adding query filter params (name=service) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:25.779206 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d72a7ea-4e49-42c2-93ca-8f6beb8e987d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:25.780588 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d72a7ea-4e49-42c2-93ca-8f6beb8e987d None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:25.780704 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 293/574] 10.222.0.16 () {56 vars in 1103 bytes} [Wed Aug 24 09:08:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:25.781395 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853935de-1cb3-4f4d-a0c8-3c84573053e6 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:25.788678 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 282/575] 10.222.0.16 () {56 vars in 1103 bytes} [Wed Aug 24 09:08:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:25.789355 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d72a7ea-4e49-42c2-93ca-8f6beb8e987d None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:25.880129 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853935de-1cb3-4f4d-a0c8-3c84573053e6 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:25.882751 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:25.889444 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d72a7ea-4e49-42c2-93ca-8f6beb8e987d None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:25.913852 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd33c916-fd41-44eb-96f6-542fcc86ce5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:25.914477 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd33c916-fd41-44eb-96f6-542fcc86ce5d None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:25.914790 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd33c916-fd41-44eb-96f6-542fcc86ce5d None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:25.915292 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd33c916-fd41-44eb-96f6-542fcc86ce5d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:25.922287 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:25.922573 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb114ee-ea24-4031-92ec-56b077e3bc49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:25.923170 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb114ee-ea24-4031-92ec-56b077e3bc49 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:25.923449 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb114ee-ea24-4031-92ec-56b077e3bc49 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:25.923753 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb114ee-ea24-4031-92ec-56b077e3bc49 None admin] PATH_INFO: `/v3/domains` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:25.924395 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb114ee-ea24-4031-92ec-56b077e3bc49 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:25.924701 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb114ee-ea24-4031-92ec-56b077e3bc49 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:25.925961 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd33c916-fd41-44eb-96f6-542fcc86ce5d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:25.926069 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb114ee-ea24-4031-92ec-56b077e3bc49 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:25.927384 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd33c916-fd41-44eb-96f6-542fcc86ce5d None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:25.938081 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 283/576] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Aug 24 09:08:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:25.939115 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb114ee-ea24-4031-92ec-56b077e3bc49 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:25.941422 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-bd33c916-fd41-44eb-96f6-542fcc86ce5d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:08:25.941422 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:08:25.941422 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:08:25.941422 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:08:25.941422 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:08:25.941422 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:08:25.941422 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:08:25.941422 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:08:25.941422 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:08:25.941422 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:08:25.941422 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:08:25.941422 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:08:25.941422 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:08:25.941422 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:08:25.941422 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:08:25.941422 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:08:25.942482 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:08:25.942482 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:08:25.942482 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:08:25.942482 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:08:25.942482 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:08:25.942482 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:08:25.942482 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:08:25.942482 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 24 09:08:25.942482 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:08:25.942482 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:08:25.942482 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:08:25.942482 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:08:25.942482 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:08:25.942482 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:08:25.942482 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:08:25.942482 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:08:25.942482 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:08:25.943361 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:08:25.943361 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:08:25.943361 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 24 09:08:25.943361 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 294/577] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 24 09:08:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:08:25.943608 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd33c916-fd41-44eb-96f6-542fcc86ce5d None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:26.041707 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb114ee-ea24-4031-92ec-56b077e3bc49 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:26.041903 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd33c916-fd41-44eb-96f6-542fcc86ce5d None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:26.074953 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066ec749-15a2-40a1-8830-ee066dbcf427 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:26.075558 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3667c9cd-bc00-4ff6-9518-64b4244fc636 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:26.075706 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066ec749-15a2-40a1-8830-ee066dbcf427 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:26.075897 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066ec749-15a2-40a1-8830-ee066dbcf427 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:26.076131 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066ec749-15a2-40a1-8830-ee066dbcf427 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:26.076228 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3667c9cd-bc00-4ff6-9518-64b4244fc636 None admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:26.076509 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3667c9cd-bc00-4ff6-9518-64b4244fc636 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:26.076783 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3667c9cd-bc00-4ff6-9518-64b4244fc636 None admin] PATH_INFO: `/v3/projects/c467e88e2fb0491bacefcf6d2ce326d9/users/1f055f7dc51d43bda9b315ca67f11d58/roles/67fd70a7d9794f7fb8faebffdec46dd2` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:26.086814 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066ec749-15a2-40a1-8830-ee066dbcf427 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:26.088357 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066ec749-15a2-40a1-8830-ee066dbcf427 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:26.091819 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3667c9cd-bc00-4ff6-9518-64b4244fc636 None admin] RBAC: Authorizing `identity:create_grant(project_id=c467e88e2fb0491bacefcf6d2ce326d9, user_id=1f055f7dc51d43bda9b315ca67f11d58, role_id=67fd70a7d9794f7fb8faebffdec46dd2)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:26.093248 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3667c9cd-bc00-4ff6-9518-64b4244fc636 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:26.102068 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-066ec749-15a2-40a1-8830-ee066dbcf427 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:08:26.102068 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:08:26.102068 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:08:26.102068 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:08:26.102068 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:08:26.102068 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:08:26.102068 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:08:26.102068 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:08:26.102068 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:08:26.102068 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:08:26.102068 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:08:26.102068 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:08:26.102068 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:08:26.102068 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:08:26.102068 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:08:26.102068 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:08:26.103189 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:08:26.103189 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:08:26.103189 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:08:26.103189 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:08:26.103189 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:08:26.103189 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:08:26.103189 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:08:26.103189 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 24 09:08:26.103189 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:08:26.103189 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:08:26.103189 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:08:26.103189 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:08:26.103189 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:08:26.103189 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:08:26.103189 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:08:26.103189 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:08:26.103189 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:08:26.104188 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:08:26.104188 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:08:26.104188 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 24 09:08:26.104188 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 295/578] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 24 09:08:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:08:26.104188 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066ec749-15a2-40a1-8830-ee066dbcf427 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:26.147060 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:26.182038 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 284/579] 10.222.0.16 () {58 vars in 1439 bytes} [Wed Aug 24 09:08:25 2022] PUT /identity/v3/projects/c467e88e2fb0491bacefcf6d2ce326d9/users/1f055f7dc51d43bda9b315ca67f11d58/roles/67fd70a7d9794f7fb8faebffdec46dd2 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:08:26.182900 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3667c9cd-bc00-4ff6-9518-64b4244fc636 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:26.187806 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:26.209846 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066ec749-15a2-40a1-8830-ee066dbcf427 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:26.251494 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f677548-198d-40e1-9044-6ea2c15c4792 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:26.252171 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f677548-198d-40e1-9044-6ea2c15c4792 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:26.252450 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f677548-198d-40e1-9044-6ea2c15c4792 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:26.252721 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f677548-198d-40e1-9044-6ea2c15c4792 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:26.264561 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f677548-198d-40e1-9044-6ea2c15c4792 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:26.266032 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f677548-198d-40e1-9044-6ea2c15c4792 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:26.282320 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-0f677548-198d-40e1-9044-6ea2c15c4792 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:08:26.282320 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:08:26.282320 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:08:26.282320 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:08:26.282320 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:08:26.282320 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:08:26.282320 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:08:26.282320 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:08:26.282320 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:08:26.282320 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:08:26.282320 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:08:26.282320 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:08:26.282320 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:08:26.282320 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:08:26.282320 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:08:26.282320 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:08:26.283544 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:08:26.283544 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:08:26.283544 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:08:26.283544 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:08:26.283544 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:08:26.283544 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:08:26.283544 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:08:26.283544 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 24 09:08:26.283544 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:08:26.283544 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:08:26.283544 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:08:26.283544 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:08:26.283544 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:08:26.283544 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:08:26.283544 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:08:26.283544 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:08:26.283544 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:08:26.284507 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:08:26.284507 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:08:26.284507 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 24 09:08:26.284507 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 296/580] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 24 09:08:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:08:26.284805 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f677548-198d-40e1-9044-6ea2c15c4792 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:26.332394 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3667c9cd-bc00-4ff6-9518-64b4244fc636 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:26.369592 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbc4b43-f759-43d0-8742-eab525f4a83a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:26.370282 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbc4b43-f759-43d0-8742-eab525f4a83a None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:26.371082 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbc4b43-f759-43d0-8742-eab525f4a83a None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:26.371357 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbc4b43-f759-43d0-8742-eab525f4a83a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:26.378281 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbc4b43-f759-43d0-8742-eab525f4a83a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:26.379803 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbc4b43-f759-43d0-8742-eab525f4a83a None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:26.391123 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-7cbc4b43-f759-43d0-8742-eab525f4a83a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:08:26.391123 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:08:26.391123 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:08:26.391123 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:08:26.391123 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:08:26.391123 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:08:26.391123 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:08:26.391123 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:08:26.391123 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:08:26.391123 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:08:26.391123 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:08:26.391123 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:08:26.391123 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 09:08:26.391123 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 09:08:26.391123 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 09:08:26.392057 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 09:08:26.392057 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:08:26.392057 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:08:26.392057 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:08:26.392057 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:08:26.392057 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:08:26.392057 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:08:26.392057 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:08:26.392057 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Aug 24 09:08:26.392057 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:08:26.392057 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:08:26.392057 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:08:26.392057 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:08:26.392057 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:08:26.392057 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:08:26.392057 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:08:26.392057 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:08:26.393186 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 09:08:26.393186 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 09:08:26.393186 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 09:08:26.393186 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 09:08:26.393186 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 09:08:26.393186 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 09:08:26.393186 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:08:26.393186 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Aug 24 09:08:26.393186 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 285/581] 10.222.0.16 () {56 vars in 1111 bytes} [Wed Aug 24 09:08:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:08:26.393673 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbc4b43-f759-43d0-8742-eab525f4a83a None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:26.406245 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:26.414324 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f677548-198d-40e1-9044-6ea2c15c4792 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:26.453377 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9c7cc3-4878-4757-b132-03a8c5bbf15b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:26.454513 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9c7cc3-4878-4757-b132-03a8c5bbf15b None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:26.454805 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9c7cc3-4878-4757-b132-03a8c5bbf15b None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:26.455081 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9c7cc3-4878-4757-b132-03a8c5bbf15b None admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:26.455909 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9c7cc3-4878-4757-b132-03a8c5bbf15b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:26.456219 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9c7cc3-4878-4757-b132-03a8c5bbf15b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:26.457908 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9c7cc3-4878-4757-b132-03a8c5bbf15b None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:26.463085 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:26.474538 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 297/582] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Aug 24 09:08:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:26.474538 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9c7cc3-4878-4757-b132-03a8c5bbf15b None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:26.509711 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbc4b43-f759-43d0-8742-eab525f4a83a None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:26.548505 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b1fe4b-3cf2-4ac7-9cf5-9dde9d4f5abb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:26.549123 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b1fe4b-3cf2-4ac7-9cf5-9dde9d4f5abb None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:26.549404 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b1fe4b-3cf2-4ac7-9cf5-9dde9d4f5abb None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:26.549678 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b1fe4b-3cf2-4ac7-9cf5-9dde9d4f5abb None admin] PATH_INFO: `/v3/domains` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:26.550415 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b1fe4b-3cf2-4ac7-9cf5-9dde9d4f5abb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:26.550725 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b1fe4b-3cf2-4ac7-9cf5-9dde9d4f5abb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:26.552128 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b1fe4b-3cf2-4ac7-9cf5-9dde9d4f5abb None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:26.568787 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 286/583] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Aug 24 09:08:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:26.568787 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b1fe4b-3cf2-4ac7-9cf5-9dde9d4f5abb None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:26.582524 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9c7cc3-4878-4757-b132-03a8c5bbf15b None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:26.616583 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4126cc2-dc0d-43b2-86ec-37e935b0d582 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:26.617154 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4126cc2-dc0d-43b2-86ec-37e935b0d582 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:26.617434 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4126cc2-dc0d-43b2-86ec-37e935b0d582 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:26.617708 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4126cc2-dc0d-43b2-86ec-37e935b0d582 None admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:26.618344 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4126cc2-dc0d-43b2-86ec-37e935b0d582 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:26.618633 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4126cc2-dc0d-43b2-86ec-37e935b0d582 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:26.620030 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4126cc2-dc0d-43b2-86ec-37e935b0d582 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:26.631660 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 298/584] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Aug 24 09:08:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:26.632728 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4126cc2-dc0d-43b2-86ec-37e935b0d582 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:26.648720 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:26.680163 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b1fe4b-3cf2-4ac7-9cf5-9dde9d4f5abb None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:26.706490 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:26.718573 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9910ee40-754a-4fc0-81fe-37fb1ca2dbb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:26.719328 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9910ee40-754a-4fc0-81fe-37fb1ca2dbb9 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:26.719822 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9910ee40-754a-4fc0-81fe-37fb1ca2dbb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:26.720177 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9910ee40-754a-4fc0-81fe-37fb1ca2dbb9 None admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:26.720938 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9910ee40-754a-4fc0-81fe-37fb1ca2dbb9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:26.721316 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9910ee40-754a-4fc0-81fe-37fb1ca2dbb9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:26.722952 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9910ee40-754a-4fc0-81fe-37fb1ca2dbb9 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:26.735205 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4126cc2-dc0d-43b2-86ec-37e935b0d582 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:26.741382 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 287/585] 10.222.0.16 () {56 vars in 1148 bytes} [Wed Aug 24 09:08:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:26.741382 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9910ee40-754a-4fc0-81fe-37fb1ca2dbb9 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:26.769932 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8635248a-a54a-4d0c-9885-d98b8450942b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:26.770566 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8635248a-a54a-4d0c-9885-d98b8450942b None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:26.770837 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8635248a-a54a-4d0c-9885-d98b8450942b None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:26.771108 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8635248a-a54a-4d0c-9885-d98b8450942b None admin] PATH_INFO: `/v3/users/glance` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:26.781049 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8635248a-a54a-4d0c-9885-d98b8450942b None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:26.782671 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8635248a-a54a-4d0c-9885-d98b8450942b None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:26.795788 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-8635248a-a54a-4d0c-9885-d98b8450942b None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 24 09:08:26.795788 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:08:26.795788 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:08:26.795788 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:08:26.795788 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:08:26.795788 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:08:26.795788 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:08:26.795788 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:08:26.795788 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:08:26.795788 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:08:26.795788 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:08:26.795788 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:08:26.795788 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 09:08:26.795788 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 09:08:26.795788 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 09:08:26.795788 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 09:08:26.796977 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:08:26.796977 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:08:26.796977 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 09:08:26.796977 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:08:26.796977 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 09:08:26.796977 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:08:26.796977 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:08:26.796977 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:08:26.796977 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:08:26.796977 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:08:26.796977 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:08:26.796977 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 24 09:08:26.796977 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:08:26.796977 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:08:26.796977 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:08:26.796977 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:08:26.796977 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:08:26.796977 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:08:26.798743 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:08:26.798743 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:08:26.798743 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 09:08:26.798743 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 09:08:26.798743 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 09:08:26.798743 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 09:08:26.798743 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 09:08:26.798743 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 09:08:26.798743 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 24 09:08:26.798743 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 24 09:08:26.801810 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 299/586] 10.222.0.16 () {56 vars in 1099 bytes} [Wed Aug 24 09:08:26 2022] GET /identity/v3/users/glance => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:08:26.802458 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8635248a-a54a-4d0c-9885-d98b8450942b None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:26.862134 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9910ee40-754a-4fc0-81fe-37fb1ca2dbb9 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:26.877130 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:26.902169 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8635248a-a54a-4d0c-9885-d98b8450942b None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:26.905687 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef014b7e-442e-4d9e-b270-109097da132e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:26.906313 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef014b7e-442e-4d9e-b270-109097da132e None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:26.906589 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef014b7e-442e-4d9e-b270-109097da132e None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:26.906858 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef014b7e-442e-4d9e-b270-109097da132e None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:26.914720 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef014b7e-442e-4d9e-b270-109097da132e None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:26.916183 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef014b7e-442e-4d9e-b270-109097da132e None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:26.928462 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-ef014b7e-442e-4d9e-b270-109097da132e None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 09:08:26.928462 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:08:26.928462 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:08:26.928462 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:08:26.928462 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:08:26.928462 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:08:26.928462 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:08:26.928462 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:08:26.928462 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:08:26.928462 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:08:26.928462 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:08:26.928462 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:08:26.928462 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 09:08:26.928462 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 09:08:26.928462 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 09:08:26.928462 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 09:08:26.929582 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:08:26.929582 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:08:26.929582 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 09:08:26.929582 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:08:26.929582 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 09:08:26.929582 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:08:26.929582 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:08:26.929582 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:08:26.929582 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:08:26.929582 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:08:26.929582 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:08:26.929582 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Aug 24 09:08:26.929582 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:08:26.929582 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:08:26.929582 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:08:26.929582 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:08:26.929582 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:08:26.929582 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:08:26.930722 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:08:26.930722 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:08:26.930722 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 09:08:26.930722 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 09:08:26.930722 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 09:08:26.930722 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 09:08:26.930722 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 09:08:26.930722 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 09:08:26.930722 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 09:08:26.930722 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Aug 24 09:08:26.930722 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 288/587] 10.222.0.16 () {56 vars in 1102 bytes} [Wed Aug 24 09:08:26 2022] GET /identity/v3/users/neutron => generated 85 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:08:26.935413 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef014b7e-442e-4d9e-b270-109097da132e None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:26.942965 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edae0e3-b50d-4652-a03c-4e3cdbdc90b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:26.943693 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edae0e3-b50d-4652-a03c-4e3cdbdc90b8 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:26.944033 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edae0e3-b50d-4652-a03c-4e3cdbdc90b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:26.944366 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edae0e3-b50d-4652-a03c-4e3cdbdc90b8 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:26.952561 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edae0e3-b50d-4652-a03c-4e3cdbdc90b8 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:26.954368 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edae0e3-b50d-4652-a03c-4e3cdbdc90b8 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:26.965364 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-0edae0e3-b50d-4652-a03c-4e3cdbdc90b8 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 09:08:26.965364 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:08:26.965364 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:08:26.965364 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:08:26.965364 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:08:26.965364 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:08:26.965364 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:08:26.965364 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:08:26.965364 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:08:26.965364 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:08:26.965364 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:08:26.965364 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:08:26.965364 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 09:08:26.965364 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 09:08:26.965364 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 09:08:26.965364 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 09:08:26.966502 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:08:26.966502 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:08:26.966502 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 09:08:26.966502 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:08:26.966502 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 09:08:26.966502 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:08:26.966502 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:08:26.966502 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:08:26.966502 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:08:26.966502 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:08:26.966502 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:08:26.966502 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 24 09:08:26.966502 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:08:26.966502 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:08:26.966502 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:08:26.966502 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:08:26.966502 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:08:26.966502 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:08:26.967460 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:08:26.967460 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:08:26.967460 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 09:08:26.967460 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 09:08:26.967460 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 09:08:26.967460 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 09:08:26.967460 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 09:08:26.967460 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 09:08:26.967460 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 09:08:26.967460 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 24 09:08:26.967460 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 300/588] 10.222.0.16 () {56 vars in 1099 bytes} [Wed Aug 24 09:08:26 2022] GET /identity/v3/users/cinder => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:08:26.968325 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edae0e3-b50d-4652-a03c-4e3cdbdc90b8 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:26.990951 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:27.047831 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef014b7e-442e-4d9e-b270-109097da132e None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:27.083669 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edae0e3-b50d-4652-a03c-4e3cdbdc90b8 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:27.086035 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e8e024-6b39-425c-8c24-88937029646b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:27.086840 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e8e024-6b39-425c-8c24-88937029646b None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:27.087271 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e8e024-6b39-425c-8c24-88937029646b None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:27.087716 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e8e024-6b39-425c-8c24-88937029646b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:27.088579 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e8e024-6b39-425c-8c24-88937029646b None admin] RBAC: Adding query filter params (role.id=67fd70a7d9794f7fb8faebffdec46dd2, scope.project.id=c467e88e2fb0491bacefcf6d2ce326d9, user.id=fdd5f0d67fe54e9fae6f4af644988d93) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:27.089066 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e8e024-6b39-425c-8c24-88937029646b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:27.090675 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e8e024-6b39-425c-8c24-88937029646b None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:27.097641 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 289/589] 10.222.0.16 () {56 vars in 1374 bytes} [Wed Aug 24 09:08:26 2022] GET /identity/v3/role_assignments?user.id=fdd5f0d67fe54e9fae6f4af644988d93&scope.project.id=c467e88e2fb0491bacefcf6d2ce326d9&role.id=67fd70a7d9794f7fb8faebffdec46dd2 => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:27.097641 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e8e024-6b39-425c-8c24-88937029646b None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:27.140841 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37be2f14-de7c-4d55-9122-6ebc248191ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:27.141506 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37be2f14-de7c-4d55-9122-6ebc248191ec None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:27.141860 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37be2f14-de7c-4d55-9122-6ebc248191ec None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:27.142209 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37be2f14-de7c-4d55-9122-6ebc248191ec None admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:27.143140 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37be2f14-de7c-4d55-9122-6ebc248191ec None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:27.143544 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37be2f14-de7c-4d55-9122-6ebc248191ec None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:27.150510 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37be2f14-de7c-4d55-9122-6ebc248191ec None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:27.150510 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:27.173741 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 301/590] 10.222.0.16 () {56 vars in 1137 bytes} [Wed Aug 24 09:08:26 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:27.174881 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37be2f14-de7c-4d55-9122-6ebc248191ec None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:27.246104 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e8e024-6b39-425c-8c24-88937029646b None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:27.260427 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:27.307536 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37be2f14-de7c-4d55-9122-6ebc248191ec None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:27.311557 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68c6de5-cd62-48e1-bcf5-bfd8868747a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:27.312446 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68c6de5-cd62-48e1-bcf5-bfd8868747a0 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:27.312956 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68c6de5-cd62-48e1-bcf5-bfd8868747a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:27.313451 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68c6de5-cd62-48e1-bcf5-bfd8868747a0 None admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:27.314392 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68c6de5-cd62-48e1-bcf5-bfd8868747a0 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:27.314987 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68c6de5-cd62-48e1-bcf5-bfd8868747a0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:27.317264 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68c6de5-cd62-48e1-bcf5-bfd8868747a0 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:27.348860 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 290/591] 10.222.0.16 () {56 vars in 1139 bytes} [Wed Aug 24 09:08:27 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:27.349928 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68c6de5-cd62-48e1-bcf5-bfd8868747a0 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:27.356920 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf728da-f464-48f1-92b5-ff6cb1494700 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:27.357804 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf728da-f464-48f1-92b5-ff6cb1494700 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:27.358297 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf728da-f464-48f1-92b5-ff6cb1494700 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:27.358802 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf728da-f464-48f1-92b5-ff6cb1494700 None admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:27.360293 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf728da-f464-48f1-92b5-ff6cb1494700 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:27.360833 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf728da-f464-48f1-92b5-ff6cb1494700 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:27.363362 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf728da-f464-48f1-92b5-ff6cb1494700 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:27.383968 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 302/592] 10.222.0.16 () {56 vars in 1137 bytes} [Wed Aug 24 09:08:27 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:27.385186 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf728da-f464-48f1-92b5-ff6cb1494700 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:27.441914 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:27.459936 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68c6de5-cd62-48e1-bcf5-bfd8868747a0 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:27.496453 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b433323-748b-4b39-838e-0fc1a97cf581 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:27.497145 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b433323-748b-4b39-838e-0fc1a97cf581 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:27.497634 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b433323-748b-4b39-838e-0fc1a97cf581 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:27.497918 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b433323-748b-4b39-838e-0fc1a97cf581 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:27.499089 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf728da-f464-48f1-92b5-ff6cb1494700 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:27.508980 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b433323-748b-4b39-838e-0fc1a97cf581 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:27.510541 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b433323-748b-4b39-838e-0fc1a97cf581 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:27.525742 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-0b433323-748b-4b39-838e-0fc1a97cf581 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:08:27.525742 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:08:27.525742 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:08:27.525742 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:08:27.525742 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:08:27.525742 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:08:27.525742 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:08:27.525742 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:08:27.525742 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:08:27.525742 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:08:27.525742 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:08:27.525742 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:08:27.525742 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:08:27.525742 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:08:27.525742 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:08:27.525742 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:08:27.527305 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:08:27.527305 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:08:27.527305 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:08:27.527305 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:08:27.527305 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:08:27.527305 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:08:27.527305 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:08:27.527305 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Aug 24 09:08:27.527305 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:08:27.527305 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:08:27.527305 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:08:27.527305 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:08:27.527305 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:08:27.527305 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:08:27.527305 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:08:27.527305 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:08:27.527305 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:08:27.528952 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:08:27.528952 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:08:27.528952 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Aug 24 09:08:27.528952 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 291/593] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 24 09:08:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:08:27.528952 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b433323-748b-4b39-838e-0fc1a97cf581 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:27.543073 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4141cbf6-6a46-4ec3-b000-68209685b6a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:27.544069 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4141cbf6-6a46-4ec3-b000-68209685b6a5 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:27.544632 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4141cbf6-6a46-4ec3-b000-68209685b6a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:27.545137 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4141cbf6-6a46-4ec3-b000-68209685b6a5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:27.548956 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:27.558056 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4141cbf6-6a46-4ec3-b000-68209685b6a5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:27.559853 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4141cbf6-6a46-4ec3-b000-68209685b6a5 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:27.575574 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-4141cbf6-6a46-4ec3-b000-68209685b6a5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:08:27.575574 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:08:27.575574 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:08:27.575574 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:08:27.575574 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:08:27.575574 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:08:27.575574 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:08:27.575574 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:08:27.575574 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:08:27.575574 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:08:27.575574 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:08:27.575574 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:08:27.575574 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:08:27.575574 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:08:27.575574 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:08:27.575574 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:08:27.576604 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:08:27.576604 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:08:27.576604 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:08:27.576604 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:08:27.576604 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:08:27.576604 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:08:27.576604 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:08:27.576604 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 24 09:08:27.576604 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:08:27.576604 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:08:27.576604 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:08:27.576604 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:08:27.576604 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:08:27.576604 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:08:27.576604 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:08:27.576604 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:08:27.576604 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:08:27.577547 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:08:27.577547 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:08:27.577547 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 24 09:08:27.581490 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 303/594] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 24 09:08:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:08:27.582577 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4141cbf6-6a46-4ec3-b000-68209685b6a5 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:27.652300 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b433323-748b-4b39-838e-0fc1a97cf581 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:27.689047 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdcf440-8ed8-49d8-afa7-02c91cacb9f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:27.689714 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdcf440-8ed8-49d8-afa7-02c91cacb9f6 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:27.689945 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdcf440-8ed8-49d8-afa7-02c91cacb9f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:27.690219 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdcf440-8ed8-49d8-afa7-02c91cacb9f6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:27.698165 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4141cbf6-6a46-4ec3-b000-68209685b6a5 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:27.700608 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:27.703206 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdcf440-8ed8-49d8-afa7-02c91cacb9f6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:27.704976 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdcf440-8ed8-49d8-afa7-02c91cacb9f6 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:27.720340 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-4cdcf440-8ed8-49d8-afa7-02c91cacb9f6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:08:27.720340 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:08:27.720340 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:08:27.720340 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:08:27.720340 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:08:27.720340 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:08:27.720340 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:08:27.720340 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:08:27.720340 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:08:27.720340 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:08:27.720340 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:08:27.720340 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:08:27.720340 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:08:27.720340 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:08:27.720340 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:08:27.720340 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:08:27.721556 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:08:27.721556 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:08:27.721556 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:08:27.721556 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:08:27.721556 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:08:27.721556 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:08:27.721556 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:08:27.721556 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Aug 24 09:08:27.721556 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:08:27.721556 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:08:27.721556 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:08:27.721556 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:08:27.721556 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:08:27.721556 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:08:27.721556 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:08:27.721556 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:08:27.721556 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:08:27.722759 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:08:27.722759 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:08:27.722759 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Aug 24 09:08:27.726325 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 292/595] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 24 09:08:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:08:27.727221 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdcf440-8ed8-49d8-afa7-02c91cacb9f6 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:27.741432 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5069ab3-ca45-4df0-a23c-fa76be555a4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:27.742351 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5069ab3-ca45-4df0-a23c-fa76be555a4b None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:27.742794 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5069ab3-ca45-4df0-a23c-fa76be555a4b None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:27.743264 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5069ab3-ca45-4df0-a23c-fa76be555a4b None admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:27.744370 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5069ab3-ca45-4df0-a23c-fa76be555a4b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:27.744968 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5069ab3-ca45-4df0-a23c-fa76be555a4b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:27.746526 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5069ab3-ca45-4df0-a23c-fa76be555a4b None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:27.760217 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 304/596] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Aug 24 09:08:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:27.762051 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5069ab3-ca45-4df0-a23c-fa76be555a4b None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:27.803334 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:27.837163 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdcf440-8ed8-49d8-afa7-02c91cacb9f6 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:27.874007 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4037639f-e94e-4840-9371-5399852d7143 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:27.874322 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5069ab3-ca45-4df0-a23c-fa76be555a4b None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:27.874618 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4037639f-e94e-4840-9371-5399852d7143 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:27.874901 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4037639f-e94e-4840-9371-5399852d7143 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:27.875177 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4037639f-e94e-4840-9371-5399852d7143 None admin] PATH_INFO: `/v3/domains` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:27.875847 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4037639f-e94e-4840-9371-5399852d7143 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:27.876153 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4037639f-e94e-4840-9371-5399852d7143 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:27.878175 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4037639f-e94e-4840-9371-5399852d7143 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:27.890673 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 293/597] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Aug 24 09:08:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:27.895670 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4037639f-e94e-4840-9371-5399852d7143 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:27.914806 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c4d927-bf33-48a4-9546-86ed7a1267a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:27.915884 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c4d927-bf33-48a4-9546-86ed7a1267a8 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:27.916320 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c4d927-bf33-48a4-9546-86ed7a1267a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:27.916746 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c4d927-bf33-48a4-9546-86ed7a1267a8 None admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:27.918133 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c4d927-bf33-48a4-9546-86ed7a1267a8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:27.918693 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c4d927-bf33-48a4-9546-86ed7a1267a8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:27.920293 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c4d927-bf33-48a4-9546-86ed7a1267a8 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:27.934243 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 305/598] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Aug 24 09:08:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:27.935284 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c4d927-bf33-48a4-9546-86ed7a1267a8 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:27.962131 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:28.013172 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4037639f-e94e-4840-9371-5399852d7143 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:28.051558 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c4d927-bf33-48a4-9546-86ed7a1267a8 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:28.061426 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:28.069761 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9711b55-cc32-4360-9784-fe0fe388a6f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:28.070826 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9711b55-cc32-4360-9784-fe0fe388a6f5 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:28.071741 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9711b55-cc32-4360-9784-fe0fe388a6f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:28.072209 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9711b55-cc32-4360-9784-fe0fe388a6f5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:28.082756 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9711b55-cc32-4360-9784-fe0fe388a6f5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:28.085351 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9711b55-cc32-4360-9784-fe0fe388a6f5 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:28.090058 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7238ea12-c4ae-4d2e-a667-743dfea5282f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:28.090702 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7238ea12-c4ae-4d2e-a667-743dfea5282f None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:28.091068 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7238ea12-c4ae-4d2e-a667-743dfea5282f None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:28.091364 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7238ea12-c4ae-4d2e-a667-743dfea5282f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:28.098438 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7238ea12-c4ae-4d2e-a667-743dfea5282f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:28.099910 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7238ea12-c4ae-4d2e-a667-743dfea5282f None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:28.100324 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-c9711b55-cc32-4360-9784-fe0fe388a6f5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:08:28.100324 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:08:28.100324 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:08:28.100324 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:08:28.100324 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:08:28.100324 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:08:28.100324 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:08:28.100324 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:08:28.100324 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:08:28.100324 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:08:28.100324 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:08:28.100324 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:08:28.100324 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 09:08:28.100324 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 09:08:28.100324 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 09:08:28.101361 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 09:08:28.101361 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:08:28.101361 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:08:28.101361 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:08:28.101361 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:08:28.101361 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:08:28.101361 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:08:28.101361 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:08:28.101361 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Aug 24 09:08:28.101361 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:08:28.101361 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:08:28.101361 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:08:28.101361 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:08:28.101361 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:08:28.101361 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:08:28.101361 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:08:28.101361 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:08:28.102447 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 09:08:28.102447 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 09:08:28.102447 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 09:08:28.102447 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 09:08:28.102447 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 09:08:28.102447 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 09:08:28.102447 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:08:28.102447 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Aug 24 09:08:28.105685 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 294/599] 10.222.0.16 () {56 vars in 1111 bytes} [Wed Aug 24 09:08:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:08:28.106612 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9711b55-cc32-4360-9784-fe0fe388a6f5 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:28.113785 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-7238ea12-c4ae-4d2e-a667-743dfea5282f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:08:28.113785 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:08:28.113785 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:08:28.113785 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:08:28.113785 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:08:28.113785 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:08:28.113785 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:08:28.113785 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:08:28.113785 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:08:28.113785 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:08:28.113785 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:08:28.113785 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:08:28.113785 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 09:08:28.113785 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 09:08:28.113785 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 09:08:28.114863 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 09:08:28.114863 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:08:28.114863 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:08:28.114863 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:08:28.114863 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:08:28.114863 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:08:28.114863 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:08:28.114863 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:08:28.114863 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 24 09:08:28.114863 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:08:28.114863 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:08:28.114863 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:08:28.114863 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:08:28.114863 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:08:28.114863 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:08:28.114863 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:08:28.114863 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:08:28.115835 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 09:08:28.115835 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 09:08:28.115835 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 09:08:28.115835 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 09:08:28.115835 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 09:08:28.115835 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 09:08:28.115835 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:08:28.115835 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 24 09:08:28.115835 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 306/600] 10.222.0.16 () {56 vars in 1111 bytes} [Wed Aug 24 09:08:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:08:28.116369 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484711e4-041d-4c6e-b318-b7ef73585c28 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:28.116443 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484711e4-041d-4c6e-b318-b7ef73585c28 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:28.116751 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484711e4-041d-4c6e-b318-b7ef73585c28 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:28.118267 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 307/601] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:08:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:08:28.119297 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484711e4-041d-4c6e-b318-b7ef73585c28 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:28.206729 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:28.223343 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9711b55-cc32-4360-9784-fe0fe388a6f5 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:28.227390 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484711e4-041d-4c6e-b318-b7ef73585c28 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:28.261295 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd05d9e-b995-4cf5-9336-02568ff8709e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:28.261927 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd05d9e-b995-4cf5-9336-02568ff8709e None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:28.262214 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd05d9e-b995-4cf5-9336-02568ff8709e None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:28.262490 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd05d9e-b995-4cf5-9336-02568ff8709e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:28.265318 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb31e619-f52c-4c14-870d-075e0d3e365a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:28.265940 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb31e619-f52c-4c14-870d-075e0d3e365a None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:28.266218 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb31e619-f52c-4c14-870d-075e0d3e365a None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:28.266486 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb31e619-f52c-4c14-870d-075e0d3e365a None admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:28.267167 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb31e619-f52c-4c14-870d-075e0d3e365a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:28.267468 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb31e619-f52c-4c14-870d-075e0d3e365a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:28.268922 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb31e619-f52c-4c14-870d-075e0d3e365a None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:28.269462 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd05d9e-b995-4cf5-9336-02568ff8709e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:28.270893 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd05d9e-b995-4cf5-9336-02568ff8709e None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:28.281155 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 308/602] 10.222.0.16 () {56 vars in 1148 bytes} [Wed Aug 24 09:08:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:28.283663 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-0fd05d9e-b995-4cf5-9336-02568ff8709e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:08:28.283663 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:08:28.283663 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:08:28.283663 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:08:28.283663 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:08:28.283663 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:08:28.283663 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:08:28.283663 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:08:28.283663 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:08:28.283663 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:08:28.283663 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:08:28.283663 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:08:28.283663 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 09:08:28.283663 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 09:08:28.283663 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 09:08:28.284549 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 09:08:28.284549 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:08:28.284549 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:08:28.284549 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:08:28.284549 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:08:28.284549 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:08:28.284549 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:08:28.284549 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:08:28.284549 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Aug 24 09:08:28.284549 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:08:28.284549 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:08:28.284549 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:08:28.284549 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:08:28.284549 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:08:28.284549 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:08:28.284549 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:08:28.284549 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:08:28.285477 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 09:08:28.285477 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 09:08:28.285477 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 09:08:28.285477 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 09:08:28.285477 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 09:08:28.285477 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 09:08:28.285477 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:08:28.285477 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Aug 24 09:08:28.287262 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc45124-1547-4190-90fc-75f4e8055d27 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:28.287548 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc45124-1547-4190-90fc-75f4e8055d27 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:28.287854 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc45124-1547-4190-90fc-75f4e8055d27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:28.289564 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 295/603] 10.222.0.16 () {56 vars in 1111 bytes} [Wed Aug 24 09:08:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:08:28.290325 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd05d9e-b995-4cf5-9336-02568ff8709e None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:28.310726 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-2cc45124-1547-4190-90fc-75f4e8055d27 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:08:28.311972 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-2cc45124-1547-4190-90fc-75f4e8055d27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:28.348118 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:28.386867 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 309/604] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:08:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:08:28.387983 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc45124-1547-4190-90fc-75f4e8055d27 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:28.409798 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd05d9e-b995-4cf5-9336-02568ff8709e None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:28.447288 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68debdb9-6e76-4e72-8543-09c63d1dc035 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:28.448054 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68debdb9-6e76-4e72-8543-09c63d1dc035 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:28.448462 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68debdb9-6e76-4e72-8543-09c63d1dc035 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:28.448831 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68debdb9-6e76-4e72-8543-09c63d1dc035 None admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:28.449626 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68debdb9-6e76-4e72-8543-09c63d1dc035 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:28.450045 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68debdb9-6e76-4e72-8543-09c63d1dc035 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:28.451562 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68debdb9-6e76-4e72-8543-09c63d1dc035 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:28.464633 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 296/605] 10.222.0.16 () {56 vars in 1148 bytes} [Wed Aug 24 09:08:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:28.465556 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68debdb9-6e76-4e72-8543-09c63d1dc035 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:28.486074 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:28.526496 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc45124-1547-4190-90fc-75f4e8055d27 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:28.565595 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad48f6c4-ce32-4d8c-b950-e2d5e8bbb90c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:28.566413 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad48f6c4-ce32-4d8c-b950-e2d5e8bbb90c None admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:28.566790 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad48f6c4-ce32-4d8c-b950-e2d5e8bbb90c None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:28.567186 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad48f6c4-ce32-4d8c-b950-e2d5e8bbb90c None admin] PATH_INFO: `/v3/projects/c467e88e2fb0491bacefcf6d2ce326d9/users/ea2666d430a646d386104addf1f42888/roles/67fd70a7d9794f7fb8faebffdec46dd2` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:28.583087 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad48f6c4-ce32-4d8c-b950-e2d5e8bbb90c None admin] RBAC: Authorizing `identity:create_grant(project_id=c467e88e2fb0491bacefcf6d2ce326d9, user_id=ea2666d430a646d386104addf1f42888, role_id=67fd70a7d9794f7fb8faebffdec46dd2)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:28.584678 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad48f6c4-ce32-4d8c-b950-e2d5e8bbb90c None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:28.589384 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68debdb9-6e76-4e72-8543-09c63d1dc035 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:28.601553 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:28.625144 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100470bf-69da-44d4-899d-d389ea0520c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:28.625774 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100470bf-69da-44d4-899d-d389ea0520c0 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:28.626177 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100470bf-69da-44d4-899d-d389ea0520c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:28.626456 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100470bf-69da-44d4-899d-d389ea0520c0 None admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:28.627141 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100470bf-69da-44d4-899d-d389ea0520c0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:28.627443 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100470bf-69da-44d4-899d-d389ea0520c0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:28.628874 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100470bf-69da-44d4-899d-d389ea0520c0 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:28.645377 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 297/606] 10.222.0.16 () {56 vars in 1148 bytes} [Wed Aug 24 09:08:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:28.645377 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100470bf-69da-44d4-899d-d389ea0520c0 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:28.664201 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 310/607] 10.222.0.16 () {58 vars in 1439 bytes} [Wed Aug 24 09:08:28 2022] PUT /identity/v3/projects/c467e88e2fb0491bacefcf6d2ce326d9/users/ea2666d430a646d386104addf1f42888/roles/67fd70a7d9794f7fb8faebffdec46dd2 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:08:28.665322 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad48f6c4-ce32-4d8c-b950-e2d5e8bbb90c None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:28.797012 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad48f6c4-ce32-4d8c-b950-e2d5e8bbb90c None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:28.799386 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100470bf-69da-44d4-899d-d389ea0520c0 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:28.826533 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:28.838422 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7286ddbd-9eb3-4f3a-ba8c-5d30e3c4f64c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:28.839752 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7286ddbd-9eb3-4f3a-ba8c-5d30e3c4f64c None admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:28.840037 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7286ddbd-9eb3-4f3a-ba8c-5d30e3c4f64c None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:28.840311 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7286ddbd-9eb3-4f3a-ba8c-5d30e3c4f64c None admin] PATH_INFO: `/v3/projects/c467e88e2fb0491bacefcf6d2ce326d9/users/8e5260e798404e8bb12b25ca8ea97052/roles/67fd70a7d9794f7fb8faebffdec46dd2` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:28.841370 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a87320-4459-48c5-8b21-dabf4638a13d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:28.842084 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a87320-4459-48c5-8b21-dabf4638a13d None admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:28.842365 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a87320-4459-48c5-8b21-dabf4638a13d None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:28.842634 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a87320-4459-48c5-8b21-dabf4638a13d None admin] PATH_INFO: `/v3/projects/c467e88e2fb0491bacefcf6d2ce326d9/users/05c32b83577c46488906fe409a790b24/roles/67fd70a7d9794f7fb8faebffdec46dd2` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:28.859821 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7286ddbd-9eb3-4f3a-ba8c-5d30e3c4f64c None admin] RBAC: Authorizing `identity:create_grant(project_id=c467e88e2fb0491bacefcf6d2ce326d9, user_id=8e5260e798404e8bb12b25ca8ea97052, role_id=67fd70a7d9794f7fb8faebffdec46dd2)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:28.860243 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a87320-4459-48c5-8b21-dabf4638a13d None admin] RBAC: Authorizing `identity:create_grant(project_id=c467e88e2fb0491bacefcf6d2ce326d9, user_id=05c32b83577c46488906fe409a790b24, role_id=67fd70a7d9794f7fb8faebffdec46dd2)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:28.861265 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7286ddbd-9eb3-4f3a-ba8c-5d30e3c4f64c None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:28.862117 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a87320-4459-48c5-8b21-dabf4638a13d None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:28.933116 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 298/608] 10.222.0.16 () {58 vars in 1439 bytes} [Wed Aug 24 09:08:28 2022] PUT /identity/v3/projects/c467e88e2fb0491bacefcf6d2ce326d9/users/8e5260e798404e8bb12b25ca8ea97052/roles/67fd70a7d9794f7fb8faebffdec46dd2 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:08:28.934368 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7286ddbd-9eb3-4f3a-ba8c-5d30e3c4f64c None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:28.963528 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 311/609] 10.222.0.16 () {58 vars in 1439 bytes} [Wed Aug 24 09:08:28 2022] PUT /identity/v3/projects/c467e88e2fb0491bacefcf6d2ce326d9/users/05c32b83577c46488906fe409a790b24/roles/67fd70a7d9794f7fb8faebffdec46dd2 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:08:28.968411 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:29.063858 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7286ddbd-9eb3-4f3a-ba8c-5d30e3c4f64c None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:29.105531 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e74e9f5-e012-4828-b23b-45fadbd163eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:29.106359 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e74e9f5-e012-4828-b23b-45fadbd163eb None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:29.106752 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e74e9f5-e012-4828-b23b-45fadbd163eb None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:29.107131 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e74e9f5-e012-4828-b23b-45fadbd163eb None admin] PATH_INFO: `/v3/roles/service` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:29.119862 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e74e9f5-e012-4828-b23b-45fadbd163eb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:08:29.125045 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-5e74e9f5-e012-4828-b23b-45fadbd163eb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 09:08:29.125045 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:08:29.125045 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:08:29.125045 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:08:29.125045 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:08:29.125045 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:08:29.125045 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:08:29.125045 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:08:29.125045 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:08:29.125045 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:08:29.125045 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:08:29.125045 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:08:29.125045 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:08:29.125045 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:08:29.125045 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:08:29.125045 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:08:29.126232 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:08:29.126232 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:08:29.126232 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:08:29.126232 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:08:29.126232 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:08:29.126232 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:08:29.126232 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:08:29.126232 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:08:29.126232 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:08:29.126232 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:08:29.126232 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:08:29.126232 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Aug 24 09:08:29.126232 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:08:29.126232 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:08:29.126232 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:08:29.126232 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:08:29.127258 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:08:29.127258 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:08:29.127258 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:08:29.127258 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:08:29.127258 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:08:29.127258 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:08:29.127258 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:08:29.127258 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:08:29.127258 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:08:29.127258 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:08:29.127258 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 09:08:29.127258 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Aug 24 09:08:29.129228 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 299/610] 10.222.0.16 () {56 vars in 1102 bytes} [Wed Aug 24 09:08:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:08:29.135043 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a87320-4459-48c5-8b21-dabf4638a13d None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:29.196481 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ca6210-2060-443e-a0dd-3ed1ee905dd8 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:29.196820 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ca6210-2060-443e-a0dd-3ed1ee905dd8 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:29.197211 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ca6210-2060-443e-a0dd-3ed1ee905dd8 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:29.199760 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 300/611] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:08:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 09:08:29.205412 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367bda84-879b-4113-8d1e-d141d8001bb0 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:29.205705 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367bda84-879b-4113-8d1e-d141d8001bb0 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:29.206215 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367bda84-879b-4113-8d1e-d141d8001bb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:29.233476 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-367bda84-879b-4113-8d1e-d141d8001bb0 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:08:29.234752 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-367bda84-879b-4113-8d1e-d141d8001bb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:29.265569 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a87320-4459-48c5-8b21-dabf4638a13d None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:29.313281 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca00c7f8-1175-46b0-a3b0-f5f8bbfaf98d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:29.314162 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca00c7f8-1175-46b0-a3b0-f5f8bbfaf98d None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:29.314614 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca00c7f8-1175-46b0-a3b0-f5f8bbfaf98d None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:29.315036 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca00c7f8-1175-46b0-a3b0-f5f8bbfaf98d None admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:29.315855 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 301/612] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:08:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 09:08:29.316056 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca00c7f8-1175-46b0-a3b0-f5f8bbfaf98d None admin] RBAC: Adding query filter params (name=service) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:29.316372 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca00c7f8-1175-46b0-a3b0-f5f8bbfaf98d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:29.317751 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca00c7f8-1175-46b0-a3b0-f5f8bbfaf98d None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:29.341601 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 312/613] 10.222.0.16 () {56 vars in 1103 bytes} [Wed Aug 24 09:08:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:29.346767 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367bda84-879b-4113-8d1e-d141d8001bb0 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:29.363550 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:29.468642 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367bda84-879b-4113-8d1e-d141d8001bb0 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:29.519446 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e13e329-dbc5-44cb-9bba-c8e51f64f9bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:29.520703 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e13e329-dbc5-44cb-9bba-c8e51f64f9bc None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:29.521305 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e13e329-dbc5-44cb-9bba-c8e51f64f9bc None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:29.521891 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e13e329-dbc5-44cb-9bba-c8e51f64f9bc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:29.536840 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e13e329-dbc5-44cb-9bba-c8e51f64f9bc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:29.538632 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e13e329-dbc5-44cb-9bba-c8e51f64f9bc None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:29.557115 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-4e13e329-dbc5-44cb-9bba-c8e51f64f9bc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:08:29.557115 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:08:29.557115 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:08:29.557115 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:08:29.557115 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:08:29.557115 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:08:29.557115 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:08:29.557115 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:08:29.557115 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:08:29.557115 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:08:29.557115 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:08:29.557115 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:08:29.557115 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:08:29.557115 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:08:29.557115 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:08:29.557115 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:08:29.558332 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:08:29.558332 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:08:29.558332 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:08:29.558332 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:08:29.558332 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:08:29.558332 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:08:29.558332 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:08:29.558332 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Aug 24 09:08:29.558332 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:08:29.558332 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:08:29.558332 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:08:29.558332 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:08:29.558332 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:08:29.558332 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:08:29.558332 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:08:29.558332 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:08:29.558332 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:08:29.559437 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:08:29.559437 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:08:29.559437 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Aug 24 09:08:29.561235 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 302/614] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 24 09:08:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:08:29.567668 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca00c7f8-1175-46b0-a3b0-f5f8bbfaf98d None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:29.608300 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e13e329-dbc5-44cb-9bba-c8e51f64f9bc None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:29.630005 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:29.687676 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:29.734353 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e13e329-dbc5-44cb-9bba-c8e51f64f9bc None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:29.772484 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1460dedc-dad6-4ea2-b093-a701f823829e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:29.773509 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1460dedc-dad6-4ea2-b093-a701f823829e None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:29.774181 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1460dedc-dad6-4ea2-b093-a701f823829e None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:29.774655 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1460dedc-dad6-4ea2-b093-a701f823829e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:29.783940 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca00c7f8-1175-46b0-a3b0-f5f8bbfaf98d None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:29.787060 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1460dedc-dad6-4ea2-b093-a701f823829e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:08:29.794114 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-1460dedc-dad6-4ea2-b093-a701f823829e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 09:08:29.794114 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:08:29.794114 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:08:29.794114 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:08:29.794114 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:08:29.794114 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:08:29.794114 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:08:29.794114 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:08:29.794114 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:08:29.794114 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:08:29.794114 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:08:29.794114 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:08:29.794114 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:08:29.794114 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:08:29.794114 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:08:29.794114 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:08:29.795072 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:08:29.795072 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:08:29.795072 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:08:29.795072 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:08:29.795072 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:08:29.795072 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:08:29.795072 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:08:29.795072 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:08:29.795072 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:08:29.795072 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:08:29.795072 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:08:29.795072 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Aug 24 09:08:29.795072 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:08:29.795072 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:08:29.795072 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:08:29.795072 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:08:29.795966 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:08:29.795966 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:08:29.795966 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:08:29.795966 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:08:29.795966 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:08:29.795966 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:08:29.795966 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:08:29.795966 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:08:29.795966 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:08:29.795966 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:08:29.795966 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 09:08:29.795966 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Aug 24 09:08:29.802486 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 303/615] 10.222.0.16 () {56 vars in 1102 bytes} [Wed Aug 24 09:08:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 09:08:29.805483 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1460dedc-dad6-4ea2-b093-a701f823829e None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:29.932028 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1460dedc-dad6-4ea2-b093-a701f823829e None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:29.932365 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89252020-9d7f-4ec4-acee-ca514dbc71af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:29.933580 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89252020-9d7f-4ec4-acee-ca514dbc71af None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:29.933858 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89252020-9d7f-4ec4-acee-ca514dbc71af None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:29.934132 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89252020-9d7f-4ec4-acee-ca514dbc71af None admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:29.934781 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89252020-9d7f-4ec4-acee-ca514dbc71af None admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:29.935078 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89252020-9d7f-4ec4-acee-ca514dbc71af None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:29.937233 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89252020-9d7f-4ec4-acee-ca514dbc71af None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:29.963328 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:29.965878 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 313/616] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Aug 24 09:08:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 400 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:29.971117 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89252020-9d7f-4ec4-acee-ca514dbc71af None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:29.973483 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c851209b-4cc7-402e-b962-dc2888f2bdf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:29.974403 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c851209b-4cc7-402e-b962-dc2888f2bdf8 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:29.974926 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c851209b-4cc7-402e-b962-dc2888f2bdf8 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:29.975428 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c851209b-4cc7-402e-b962-dc2888f2bdf8 None admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:29.976306 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c851209b-4cc7-402e-b962-dc2888f2bdf8 None admin] RBAC: Adding query filter params (name=service) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:29.976815 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c851209b-4cc7-402e-b962-dc2888f2bdf8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:29.978574 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c851209b-4cc7-402e-b962-dc2888f2bdf8 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:29.988737 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 304/617] 10.222.0.16 () {56 vars in 1103 bytes} [Wed Aug 24 09:08:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:29.994832 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c851209b-4cc7-402e-b962-dc2888f2bdf8 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:30.128835 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c851209b-4cc7-402e-b962-dc2888f2bdf8 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:30.170526 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:30.181235 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89252020-9d7f-4ec4-acee-ca514dbc71af None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:30.188445 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0016016c-a0ac-43be-9e36-17b6649c02a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:30.189027 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0016016c-a0ac-43be-9e36-17b6649c02a1 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:30.189306 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0016016c-a0ac-43be-9e36-17b6649c02a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:30.189574 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0016016c-a0ac-43be-9e36-17b6649c02a1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:30.205865 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0016016c-a0ac-43be-9e36-17b6649c02a1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:30.207368 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0016016c-a0ac-43be-9e36-17b6649c02a1 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:30.223075 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-0016016c-a0ac-43be-9e36-17b6649c02a1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:08:30.223075 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:08:30.223075 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:08:30.223075 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:08:30.223075 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:08:30.223075 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:08:30.223075 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:08:30.223075 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:08:30.223075 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:08:30.223075 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:08:30.223075 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:08:30.223075 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:08:30.223075 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:08:30.223075 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:08:30.223075 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:08:30.223075 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:08:30.224398 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:08:30.224398 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:08:30.224398 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:08:30.224398 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:08:30.224398 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:08:30.224398 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:08:30.224398 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:08:30.224398 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Aug 24 09:08:30.224398 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:08:30.224398 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:08:30.224398 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:08:30.224398 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:08:30.224398 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:08:30.224398 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:08:30.224398 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:08:30.224398 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:08:30.224398 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:08:30.225488 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:08:30.225488 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:08:30.225488 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Aug 24 09:08:30.225488 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 305/618] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 24 09:08:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:08:30.231074 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0016016c-a0ac-43be-9e36-17b6649c02a1 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:30.251644 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea55c88f-aa34-4c77-afe4-c9c8b166aed1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:30.252524 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea55c88f-aa34-4c77-afe4-c9c8b166aed1 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:30.252960 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea55c88f-aa34-4c77-afe4-c9c8b166aed1 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:30.253625 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea55c88f-aa34-4c77-afe4-c9c8b166aed1 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:30.264350 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea55c88f-aa34-4c77-afe4-c9c8b166aed1 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:30.267276 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea55c88f-aa34-4c77-afe4-c9c8b166aed1 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:30.288337 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-ea55c88f-aa34-4c77-afe4-c9c8b166aed1 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 09:08:30.288337 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:08:30.288337 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:08:30.288337 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:08:30.288337 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:08:30.288337 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:08:30.288337 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:08:30.288337 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:08:30.288337 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:08:30.288337 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:08:30.288337 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:08:30.288337 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:08:30.288337 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 09:08:30.288337 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 09:08:30.288337 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 09:08:30.288337 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 09:08:30.289573 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:08:30.289573 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:08:30.289573 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 09:08:30.289573 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:08:30.289573 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 09:08:30.289573 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:08:30.289573 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:08:30.289573 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:08:30.289573 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:08:30.289573 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:08:30.289573 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:08:30.289573 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 24 09:08:30.289573 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:08:30.289573 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:08:30.289573 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:08:30.289573 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:08:30.289573 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:08:30.289573 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:08:30.290779 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:08:30.290779 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:08:30.290779 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 09:08:30.290779 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 09:08:30.290779 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 09:08:30.290779 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 09:08:30.290779 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 09:08:30.290779 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 09:08:30.290779 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 09:08:30.290779 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 24 09:08:30.296704 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 314/619] 10.222.0.16 () {56 vars in 1093 bytes} [Wed Aug 24 09:08:29 2022] GET /identity/v3/users/nova => generated 82 bytes in 326 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:08:30.306469 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea55c88f-aa34-4c77-afe4-c9c8b166aed1 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:30.307650 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:30.441925 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0016016c-a0ac-43be-9e36-17b6649c02a1 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:30.501706 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a814bf10-105e-457f-ac58-3a14c56fa88f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:30.502423 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a814bf10-105e-457f-ac58-3a14c56fa88f None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:30.502701 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a814bf10-105e-457f-ac58-3a14c56fa88f None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:30.502971 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a814bf10-105e-457f-ac58-3a14c56fa88f None admin] PATH_INFO: `/v3/domains` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:30.503641 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a814bf10-105e-457f-ac58-3a14c56fa88f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:30.503949 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a814bf10-105e-457f-ac58-3a14c56fa88f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:30.505315 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a814bf10-105e-457f-ac58-3a14c56fa88f None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:30.512883 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea55c88f-aa34-4c77-afe4-c9c8b166aed1 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:30.540312 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 306/620] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Aug 24 09:08:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:08:30.545649 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a814bf10-105e-457f-ac58-3a14c56fa88f None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:30.587757 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d595d93-c3bd-4acc-81da-2274f4586264 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:30.588417 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d595d93-c3bd-4acc-81da-2274f4586264 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:30.588694 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d595d93-c3bd-4acc-81da-2274f4586264 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:30.589605 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d595d93-c3bd-4acc-81da-2274f4586264 None admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:30.590342 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d595d93-c3bd-4acc-81da-2274f4586264 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:30.590644 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d595d93-c3bd-4acc-81da-2274f4586264 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:30.592105 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d595d93-c3bd-4acc-81da-2274f4586264 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:30.626104 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 315/621] 10.222.0.16 () {56 vars in 1133 bytes} [Wed Aug 24 09:08:30 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:30.641007 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d595d93-c3bd-4acc-81da-2274f4586264 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:30.677016 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:30.683245 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a814bf10-105e-457f-ac58-3a14c56fa88f None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:30.702409 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:30.721866 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e55db59-8420-45df-a982-843c62a8c65f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:30.722497 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e55db59-8420-45df-a982-843c62a8c65f None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:30.722774 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e55db59-8420-45df-a982-843c62a8c65f None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:30.723041 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e55db59-8420-45df-a982-843c62a8c65f None admin] PATH_INFO: `/v3/users/swift` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:30.731487 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e55db59-8420-45df-a982-843c62a8c65f None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:30.733391 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e55db59-8420-45df-a982-843c62a8c65f None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:30.746928 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-8e55db59-8420-45df-a982-843c62a8c65f None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 09:08:30.746928 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:08:30.746928 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:08:30.746928 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:08:30.746928 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:08:30.746928 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:08:30.746928 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:08:30.746928 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:08:30.746928 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:08:30.746928 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:08:30.746928 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:08:30.746928 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:08:30.746928 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 09:08:30.746928 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 09:08:30.746928 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 09:08:30.746928 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 09:08:30.747938 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:08:30.747938 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:08:30.747938 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 09:08:30.747938 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:08:30.747938 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 09:08:30.747938 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:08:30.747938 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:08:30.747938 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:08:30.747938 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:08:30.747938 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:08:30.747938 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:08:30.747938 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Aug 24 09:08:30.747938 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:08:30.747938 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:08:30.747938 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:08:30.747938 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:08:30.747938 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:08:30.747938 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:08:30.748907 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:08:30.748907 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:08:30.748907 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 09:08:30.748907 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 09:08:30.748907 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 09:08:30.748907 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 09:08:30.748907 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 09:08:30.748907 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 09:08:30.748907 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 09:08:30.748907 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Aug 24 09:08:30.748907 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 307/622] 10.222.0.16 () {56 vars in 1096 bytes} [Wed Aug 24 09:08:30 2022] GET /identity/v3/users/swift => generated 83 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:08:30.756695 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e55db59-8420-45df-a982-843c62a8c65f None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:30.857264 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d595d93-c3bd-4acc-81da-2274f4586264 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:30.896836 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6112fac0-ca72-4024-917b-7bae09f82927 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:30.897829 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6112fac0-ca72-4024-917b-7bae09f82927 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:30.899012 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6112fac0-ca72-4024-917b-7bae09f82927 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:30.900075 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6112fac0-ca72-4024-917b-7bae09f82927 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:30.950255 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e55db59-8420-45df-a982-843c62a8c65f None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:30.974092 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6112fac0-ca72-4024-917b-7bae09f82927 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:30.975798 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6112fac0-ca72-4024-917b-7bae09f82927 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:30.991442 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:30.996546 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14273ea2-d5ad-493f-876d-66cfce74e99e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:30.997467 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14273ea2-d5ad-493f-876d-66cfce74e99e None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:30.997824 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14273ea2-d5ad-493f-876d-66cfce74e99e None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:30.998102 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14273ea2-d5ad-493f-876d-66cfce74e99e None admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:30.998854 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14273ea2-d5ad-493f-876d-66cfce74e99e None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:30.999164 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14273ea2-d5ad-493f-876d-66cfce74e99e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:31.001161 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14273ea2-d5ad-493f-876d-66cfce74e99e None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:31.023886 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 308/623] 10.222.0.16 () {56 vars in 1135 bytes} [Wed Aug 24 09:08:30 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:31.026103 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79a5375-d594-4a04-b604-a0f4c8e8e5dd None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:31.027098 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79a5375-d594-4a04-b604-a0f4c8e8e5dd None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:31.027516 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79a5375-d594-4a04-b604-a0f4c8e8e5dd None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:31.031283 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 309/624] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:08:31 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:08:31.033055 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79a5375-d594-4a04-b604-a0f4c8e8e5dd None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:31.056709 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:31.093925 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 316/625] 10.222.0.16 () {58 vars in 1282 bytes} [Wed Aug 24 09:08:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 453 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 09:08:31.095657 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b640e8-4e80-44fd-9a7c-3df687df0347 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:31.096159 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b640e8-4e80-44fd-9a7c-3df687df0347 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:31.096683 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b640e8-4e80-44fd-9a7c-3df687df0347 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:31.117918 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-34b640e8-4e80-44fd-9a7c-3df687df0347 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:08:31.119254 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-34b640e8-4e80-44fd-9a7c-3df687df0347 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:31.172717 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79a5375-d594-4a04-b604-a0f4c8e8e5dd None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:31.198908 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 317/626] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:08:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:08:31.200878 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b640e8-4e80-44fd-9a7c-3df687df0347 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:31.236250 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a39447-b74a-4216-8a7f-9097176b85c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:31.237182 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a39447-b74a-4216-8a7f-9097176b85c5 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:31.237646 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a39447-b74a-4216-8a7f-9097176b85c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:31.238116 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a39447-b74a-4216-8a7f-9097176b85c5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:31.255139 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a39447-b74a-4216-8a7f-9097176b85c5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:31.257918 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a39447-b74a-4216-8a7f-9097176b85c5 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:31.277660 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-51a39447-b74a-4216-8a7f-9097176b85c5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:08:31.277660 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:08:31.277660 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:08:31.277660 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:08:31.277660 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:08:31.277660 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:08:31.277660 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:08:31.277660 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:08:31.277660 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:08:31.277660 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:08:31.277660 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:08:31.277660 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:08:31.277660 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:08:31.277660 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:08:31.277660 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:08:31.277660 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:08:31.278686 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:08:31.278686 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:08:31.278686 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:08:31.278686 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:08:31.278686 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:08:31.278686 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:08:31.278686 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:08:31.278686 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Aug 24 09:08:31.278686 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:08:31.278686 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:08:31.278686 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:08:31.278686 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:08:31.278686 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:08:31.278686 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:08:31.278686 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:08:31.278686 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:08:31.278686 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:08:31.284180 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:08:31.284180 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:08:31.284180 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Aug 24 09:08:31.284180 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 310/627] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 24 09:08:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:08:31.284180 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dfa585-88bd-49c4-9a66-2b7740e5400d None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:31.284180 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dfa585-88bd-49c4-9a66-2b7740e5400d None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:31.284180 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dfa585-88bd-49c4-9a66-2b7740e5400d None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:31.284180 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 311/628] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:08:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:08:31.287135 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98dfa585-88bd-49c4-9a66-2b7740e5400d None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:31.300261 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:31.356757 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b640e8-4e80-44fd-9a7c-3df687df0347 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:31.392767 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:31.394878 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6618a9b-bca1-47b7-8c0d-eea159ba5128 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:31.395388 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6618a9b-bca1-47b7-8c0d-eea159ba5128 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:31.395670 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6618a9b-bca1-47b7-8c0d-eea159ba5128 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:31.395961 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6618a9b-bca1-47b7-8c0d-eea159ba5128 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:31.406947 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6618a9b-bca1-47b7-8c0d-eea159ba5128 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:31.408427 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6618a9b-bca1-47b7-8c0d-eea159ba5128 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:31.423709 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-f6618a9b-bca1-47b7-8c0d-eea159ba5128 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:08:31.423709 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:08:31.423709 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:08:31.423709 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:08:31.423709 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:08:31.423709 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:08:31.423709 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:08:31.423709 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:08:31.423709 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:08:31.423709 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:08:31.423709 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:08:31.423709 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:08:31.423709 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:08:31.423709 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:08:31.423709 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:08:31.423709 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:08:31.424668 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:08:31.424668 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:08:31.424668 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:08:31.424668 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:08:31.424668 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:08:31.424668 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:08:31.424668 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:08:31.424668 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 24 09:08:31.424668 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:08:31.424668 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:08:31.424668 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:08:31.424668 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:08:31.424668 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:08:31.424668 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:08:31.424668 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:08:31.424668 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:08:31.424668 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:08:31.425643 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:08:31.425643 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:08:31.425643 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 24 09:08:31.425643 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 318/629] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 24 09:08:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:08:31.426275 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534db49e-30c7-4fb3-a197-b0930ad98b95 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:31.426275 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534db49e-30c7-4fb3-a197-b0930ad98b95 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:31.426518 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534db49e-30c7-4fb3-a197-b0930ad98b95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:31.436058 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98dfa585-88bd-49c4-9a66-2b7740e5400d None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:31.444438 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-534db49e-30c7-4fb3-a197-b0930ad98b95 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:08:31.445635 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-534db49e-30c7-4fb3-a197-b0930ad98b95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:31.472738 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87487b3-1e79-4c34-b02a-5d435fac2380 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:31.473362 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87487b3-1e79-4c34-b02a-5d435fac2380 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:31.473643 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87487b3-1e79-4c34-b02a-5d435fac2380 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:31.473916 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87487b3-1e79-4c34-b02a-5d435fac2380 None admin] PATH_INFO: `/v3/domains` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:31.474556 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87487b3-1e79-4c34-b02a-5d435fac2380 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:31.474877 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87487b3-1e79-4c34-b02a-5d435fac2380 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:31.476259 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87487b3-1e79-4c34-b02a-5d435fac2380 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:31.488899 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 312/630] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Aug 24 09:08:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:31.490011 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884e33b4-a2b3-4a04-9138-7938e11bc01e None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:31.490298 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884e33b4-a2b3-4a04-9138-7938e11bc01e None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:31.490578 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884e33b4-a2b3-4a04-9138-7938e11bc01e None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:31.493865 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 313/631] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:08:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:08:31.493865 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884e33b4-a2b3-4a04-9138-7938e11bc01e None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:31.520461 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 319/632] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:08:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:08:31.521302 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534db49e-30c7-4fb3-a197-b0930ad98b95 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:31.546079 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:31.608485 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884e33b4-a2b3-4a04-9138-7938e11bc01e None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:31.627229 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534db49e-30c7-4fb3-a197-b0930ad98b95 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:31.635605 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:31.644457 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388c723d-cb4c-410f-b81a-d89847b0737e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:31.645068 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388c723d-cb4c-410f-b81a-d89847b0737e None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:31.645349 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388c723d-cb4c-410f-b81a-d89847b0737e None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:31.645624 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388c723d-cb4c-410f-b81a-d89847b0737e None admin] PATH_INFO: `/v3/domains` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:31.646265 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388c723d-cb4c-410f-b81a-d89847b0737e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:31.646569 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388c723d-cb4c-410f-b81a-d89847b0737e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:31.647978 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388c723d-cb4c-410f-b81a-d89847b0737e None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:31.659131 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 314/633] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Aug 24 09:08:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:31.660338 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fa28d8-014b-4356-b5e3-dd314015ffca None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:31.660623 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fa28d8-014b-4356-b5e3-dd314015ffca None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:31.660895 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fa28d8-014b-4356-b5e3-dd314015ffca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:31.666517 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30da7433-2769-424f-b0d0-582303dc831d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:31.670545 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30da7433-2769-424f-b0d0-582303dc831d None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:31.670545 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30da7433-2769-424f-b0d0-582303dc831d None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:31.670545 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30da7433-2769-424f-b0d0-582303dc831d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:31.675707 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30da7433-2769-424f-b0d0-582303dc831d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:31.677143 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30da7433-2769-424f-b0d0-582303dc831d None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:31.680037 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-c8fa28d8-014b-4356-b5e3-dd314015ffca None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:08:31.681219 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-c8fa28d8-014b-4356-b5e3-dd314015ffca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:31.689056 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-30da7433-2769-424f-b0d0-582303dc831d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:08:31.689056 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:08:31.689056 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:08:31.689056 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:08:31.689056 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:08:31.689056 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:08:31.689056 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:08:31.689056 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:08:31.689056 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:08:31.689056 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:08:31.689056 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:08:31.689056 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:08:31.689056 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 09:08:31.689056 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 09:08:31.689056 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 09:08:31.689970 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 09:08:31.689970 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:08:31.689970 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:08:31.689970 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:08:31.689970 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:08:31.689970 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:08:31.689970 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:08:31.689970 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:08:31.689970 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 24 09:08:31.689970 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:08:31.689970 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:08:31.689970 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:08:31.689970 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:08:31.689970 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:08:31.689970 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:08:31.689970 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:08:31.689970 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:08:31.690867 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 09:08:31.690867 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 09:08:31.690867 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 09:08:31.690867 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 09:08:31.690867 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 09:08:31.690867 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 09:08:31.690867 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:08:31.690867 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 24 09:08:31.690867 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 320/634] 10.222.0.16 () {56 vars in 1111 bytes} [Wed Aug 24 09:08:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:08:31.691359 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30da7433-2769-424f-b0d0-582303dc831d None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:31.758737 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 315/635] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:08:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:08:31.759920 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fa28d8-014b-4356-b5e3-dd314015ffca None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:31.790459 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:31.808498 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30da7433-2769-424f-b0d0-582303dc831d None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:31.844700 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c1443e-706f-4c92-988e-9e8db8082a3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:31.845470 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c1443e-706f-4c92-988e-9e8db8082a3e None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:31.845751 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c1443e-706f-4c92-988e-9e8db8082a3e None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:31.846023 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c1443e-706f-4c92-988e-9e8db8082a3e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:31.856539 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c1443e-706f-4c92-988e-9e8db8082a3e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:08:31.861147 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-05c1443e-706f-4c92-988e-9e8db8082a3e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 09:08:31.861147 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:08:31.861147 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:08:31.861147 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:08:31.861147 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:08:31.861147 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:08:31.861147 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:08:31.861147 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:08:31.861147 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:08:31.861147 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:08:31.861147 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:08:31.861147 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:08:31.861147 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:08:31.861147 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:08:31.861147 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:08:31.861147 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:08:31.862258 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:08:31.862258 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:08:31.862258 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:08:31.862258 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:08:31.862258 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:08:31.862258 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:08:31.862258 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:08:31.862258 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:08:31.862258 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:08:31.862258 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:08:31.862258 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:08:31.862258 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 24 09:08:31.862258 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:08:31.862258 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:08:31.862258 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:08:31.862258 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:08:31.864492 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:08:31.864492 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:08:31.864492 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:08:31.864492 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:08:31.864492 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:08:31.864492 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:08:31.864492 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:08:31.864492 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:08:31.864492 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:08:31.864492 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:08:31.864492 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 09:08:31.864492 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 24 09:08:31.864492 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 321/636] 10.222.0.16 () {56 vars in 1102 bytes} [Wed Aug 24 09:08:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:08:31.864492 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c1443e-706f-4c92-988e-9e8db8082a3e None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:31.871681 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fa28d8-014b-4356-b5e3-dd314015ffca None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:31.889404 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:31.906813 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1889b08-cd27-420d-b624-72657bd475c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:31.907437 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1889b08-cd27-420d-b624-72657bd475c5 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:31.907801 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1889b08-cd27-420d-b624-72657bd475c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:31.908115 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1889b08-cd27-420d-b624-72657bd475c5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:31.914636 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1889b08-cd27-420d-b624-72657bd475c5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:31.916101 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1889b08-cd27-420d-b624-72657bd475c5 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:31.928991 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-f1889b08-cd27-420d-b624-72657bd475c5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:08:31.928991 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:08:31.928991 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:08:31.928991 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:08:31.928991 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:08:31.928991 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:08:31.928991 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:08:31.928991 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:08:31.928991 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:08:31.928991 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:08:31.928991 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:08:31.928991 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:08:31.928991 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 09:08:31.928991 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 09:08:31.928991 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 09:08:31.930101 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 09:08:31.930101 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:08:31.930101 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:08:31.930101 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:08:31.930101 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:08:31.930101 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:08:31.930101 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:08:31.930101 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:08:31.930101 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Aug 24 09:08:31.930101 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:08:31.930101 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:08:31.930101 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:08:31.930101 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:08:31.930101 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:08:31.930101 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:08:31.930101 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:08:31.930101 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:08:31.931218 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 09:08:31.931218 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 09:08:31.931218 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 09:08:31.931218 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 09:08:31.931218 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 09:08:31.931218 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 09:08:31.931218 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:08:31.931218 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Aug 24 09:08:31.935204 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 316/637] 10.222.0.16 () {56 vars in 1111 bytes} [Wed Aug 24 09:08:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:08:31.936131 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1889b08-cd27-420d-b624-72657bd475c5 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:31.975692 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c1443e-706f-4c92-988e-9e8db8082a3e None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:32.009998 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c5507b-24d8-40c0-aa74-72b3580e501b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:32.011248 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c5507b-24d8-40c0-aa74-72b3580e501b None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:32.011634 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c5507b-24d8-40c0-aa74-72b3580e501b None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:32.012010 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c5507b-24d8-40c0-aa74-72b3580e501b None admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:32.012798 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c5507b-24d8-40c0-aa74-72b3580e501b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:32.013284 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c5507b-24d8-40c0-aa74-72b3580e501b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:32.014810 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c5507b-24d8-40c0-aa74-72b3580e501b None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:32.031522 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 322/638] 10.222.0.16 () {56 vars in 1148 bytes} [Wed Aug 24 09:08:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:32.031522 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c5507b-24d8-40c0-aa74-72b3580e501b None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:32.034484 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:32.040825 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1889b08-cd27-420d-b624-72657bd475c5 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:32.075390 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6214c528-0ca3-4f1b-b05a-5f086f92de2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:32.076049 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6214c528-0ca3-4f1b-b05a-5f086f92de2a None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:32.076335 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6214c528-0ca3-4f1b-b05a-5f086f92de2a None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:32.076609 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6214c528-0ca3-4f1b-b05a-5f086f92de2a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:32.087396 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6214c528-0ca3-4f1b-b05a-5f086f92de2a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:08:32.092136 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-6214c528-0ca3-4f1b-b05a-5f086f92de2a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 09:08:32.092136 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:08:32.092136 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:08:32.092136 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:08:32.092136 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:08:32.092136 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:08:32.092136 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:08:32.092136 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:08:32.092136 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:08:32.092136 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:08:32.092136 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:08:32.092136 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:08:32.092136 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:08:32.092136 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:08:32.092136 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:08:32.092136 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:08:32.093307 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:08:32.093307 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:08:32.093307 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:08:32.093307 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:08:32.093307 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:08:32.093307 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:08:32.093307 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:08:32.093307 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:08:32.093307 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:08:32.093307 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:08:32.093307 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:08:32.093307 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Aug 24 09:08:32.093307 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:08:32.093307 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:08:32.093307 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:08:32.093307 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:08:32.094419 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:08:32.094419 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:08:32.094419 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:08:32.094419 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:08:32.094419 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:08:32.094419 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:08:32.094419 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:08:32.094419 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:08:32.094419 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:08:32.094419 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:08:32.094419 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 09:08:32.094419 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Aug 24 09:08:32.094419 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 317/639] 10.222.0.16 () {56 vars in 1102 bytes} [Wed Aug 24 09:08:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:08:32.094419 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6214c528-0ca3-4f1b-b05a-5f086f92de2a None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:32.129013 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:32.134776 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c5507b-24d8-40c0-aa74-72b3580e501b None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:32.168261 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34234ab-72a4-4e52-b797-43ede3d00035 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:32.168887 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34234ab-72a4-4e52-b797-43ede3d00035 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:32.169164 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34234ab-72a4-4e52-b797-43ede3d00035 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:32.169431 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34234ab-72a4-4e52-b797-43ede3d00035 None admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:32.170135 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34234ab-72a4-4e52-b797-43ede3d00035 None admin] RBAC: Adding query filter params (name=service) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:32.170450 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34234ab-72a4-4e52-b797-43ede3d00035 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:32.171850 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34234ab-72a4-4e52-b797-43ede3d00035 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:32.179460 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 323/640] 10.222.0.16 () {56 vars in 1103 bytes} [Wed Aug 24 09:08:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:32.180261 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34234ab-72a4-4e52-b797-43ede3d00035 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:32.194646 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6214c528-0ca3-4f1b-b05a-5f086f92de2a None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:32.231479 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641e215c-6eef-4cfb-a43a-6c3e8900aa35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:32.232130 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641e215c-6eef-4cfb-a43a-6c3e8900aa35 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:32.232409 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641e215c-6eef-4cfb-a43a-6c3e8900aa35 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:32.232683 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641e215c-6eef-4cfb-a43a-6c3e8900aa35 None admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:32.233375 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641e215c-6eef-4cfb-a43a-6c3e8900aa35 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:32.233679 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641e215c-6eef-4cfb-a43a-6c3e8900aa35 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:32.235147 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641e215c-6eef-4cfb-a43a-6c3e8900aa35 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:32.246905 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 318/641] 10.222.0.16 () {56 vars in 1148 bytes} [Wed Aug 24 09:08:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:32.247725 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641e215c-6eef-4cfb-a43a-6c3e8900aa35 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:32.262041 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:32.296530 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34234ab-72a4-4e52-b797-43ede3d00035 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:32.331216 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f05ef1-53f5-4f13-adb6-359a2c740ce1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:32.331883 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f05ef1-53f5-4f13-adb6-359a2c740ce1 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:32.332163 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f05ef1-53f5-4f13-adb6-359a2c740ce1 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:32.332436 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f05ef1-53f5-4f13-adb6-359a2c740ce1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:32.343408 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f05ef1-53f5-4f13-adb6-359a2c740ce1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:08:32.345763 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:32.348713 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-34f05ef1-53f5-4f13-adb6-359a2c740ce1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 09:08:32.348713 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:08:32.348713 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:08:32.348713 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:08:32.348713 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:08:32.348713 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:08:32.348713 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:08:32.348713 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:08:32.348713 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:08:32.348713 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:08:32.348713 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:08:32.348713 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:08:32.348713 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:08:32.348713 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:08:32.348713 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:08:32.348713 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:08:32.349687 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:08:32.349687 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:08:32.349687 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:08:32.349687 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:08:32.349687 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:08:32.349687 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:08:32.349687 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:08:32.349687 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:08:32.349687 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:08:32.349687 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:08:32.349687 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:08:32.349687 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 24 09:08:32.349687 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:08:32.349687 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:08:32.349687 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:08:32.349687 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:08:32.350673 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:08:32.350673 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:08:32.350673 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:08:32.350673 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:08:32.350673 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:08:32.350673 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:08:32.350673 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:08:32.350673 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:08:32.350673 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:08:32.350673 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:08:32.350673 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 09:08:32.350673 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 24 09:08:32.350673 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 324/642] 10.222.0.16 () {56 vars in 1102 bytes} [Wed Aug 24 09:08:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:08:32.351536 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f05ef1-53f5-4f13-adb6-359a2c740ce1 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:32.356397 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641e215c-6eef-4cfb-a43a-6c3e8900aa35 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:32.390916 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435e7a70-e28c-49ce-ae5f-402bfc492fe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:32.391565 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435e7a70-e28c-49ce-ae5f-402bfc492fe4 None admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:32.391985 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435e7a70-e28c-49ce-ae5f-402bfc492fe4 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:32.392263 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435e7a70-e28c-49ce-ae5f-402bfc492fe4 None admin] PATH_INFO: `/v3/projects/c467e88e2fb0491bacefcf6d2ce326d9/users/fdd5f0d67fe54e9fae6f4af644988d93/roles/67fd70a7d9794f7fb8faebffdec46dd2` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:32.406957 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435e7a70-e28c-49ce-ae5f-402bfc492fe4 None admin] RBAC: Authorizing `identity:create_grant(project_id=c467e88e2fb0491bacefcf6d2ce326d9, user_id=fdd5f0d67fe54e9fae6f4af644988d93, role_id=67fd70a7d9794f7fb8faebffdec46dd2)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:32.408416 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435e7a70-e28c-49ce-ae5f-402bfc492fe4 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:32.448553 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f05ef1-53f5-4f13-adb6-359a2c740ce1 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:32.472720 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:32.478509 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 319/643] 10.222.0.16 () {58 vars in 1439 bytes} [Wed Aug 24 09:08:32 2022] PUT /identity/v3/projects/c467e88e2fb0491bacefcf6d2ce326d9/users/fdd5f0d67fe54e9fae6f4af644988d93/roles/67fd70a7d9794f7fb8faebffdec46dd2 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:08:32.479978 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435e7a70-e28c-49ce-ae5f-402bfc492fe4 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:32.489509 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceefa505-cff8-42f4-90e8-60f1a480ea4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:32.490241 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceefa505-cff8-42f4-90e8-60f1a480ea4d None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:32.490591 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceefa505-cff8-42f4-90e8-60f1a480ea4d None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:32.490932 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceefa505-cff8-42f4-90e8-60f1a480ea4d None admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:32.491747 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceefa505-cff8-42f4-90e8-60f1a480ea4d None admin] RBAC: Adding query filter params (name=service) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:32.492124 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceefa505-cff8-42f4-90e8-60f1a480ea4d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:32.493627 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceefa505-cff8-42f4-90e8-60f1a480ea4d None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:32.504244 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 325/644] 10.222.0.16 () {56 vars in 1103 bytes} [Wed Aug 24 09:08:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:32.505402 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceefa505-cff8-42f4-90e8-60f1a480ea4d None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:32.618228 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceefa505-cff8-42f4-90e8-60f1a480ea4d None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:32.627626 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435e7a70-e28c-49ce-ae5f-402bfc492fe4 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:32.658813 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdc4707-1673-4860-a340-996d10ce3ba7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:32.660273 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdc4707-1673-4860-a340-996d10ce3ba7 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:32.660714 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdc4707-1673-4860-a340-996d10ce3ba7 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:32.661136 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdc4707-1673-4860-a340-996d10ce3ba7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:32.662032 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdc4707-1673-4860-a340-996d10ce3ba7 None admin] RBAC: Adding query filter params (role.id=67fd70a7d9794f7fb8faebffdec46dd2, scope.project.id=c467e88e2fb0491bacefcf6d2ce326d9, user.id=1f055f7dc51d43bda9b315ca67f11d58) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:32.662489 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdc4707-1673-4860-a340-996d10ce3ba7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:32.664215 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdc4707-1673-4860-a340-996d10ce3ba7 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:32.665017 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249b77bd-8901-4e64-a631-2257e63e2984 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:32.665644 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249b77bd-8901-4e64-a631-2257e63e2984 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:32.665923 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249b77bd-8901-4e64-a631-2257e63e2984 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:32.666195 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249b77bd-8901-4e64-a631-2257e63e2984 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:32.673449 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 326/645] 10.222.0.16 () {56 vars in 1374 bytes} [Wed Aug 24 09:08:32 2022] GET /identity/v3/role_assignments?user.id=1f055f7dc51d43bda9b315ca67f11d58&scope.project.id=c467e88e2fb0491bacefcf6d2ce326d9&role.id=67fd70a7d9794f7fb8faebffdec46dd2 => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:32.674944 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdc4707-1673-4860-a340-996d10ce3ba7 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:32.678387 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249b77bd-8901-4e64-a631-2257e63e2984 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:32.680111 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249b77bd-8901-4e64-a631-2257e63e2984 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:32.688366 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:32.697709 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-249b77bd-8901-4e64-a631-2257e63e2984 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:08:32.697709 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:08:32.697709 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:08:32.697709 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:08:32.697709 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:08:32.697709 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:08:32.697709 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:08:32.697709 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:08:32.697709 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:08:32.697709 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:08:32.697709 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:08:32.697709 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:08:32.697709 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:08:32.697709 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:08:32.697709 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:08:32.697709 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:08:32.704702 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:08:32.704702 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:08:32.704702 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:08:32.704702 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:08:32.704702 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:08:32.704702 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:08:32.704702 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:08:32.704702 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Aug 24 09:08:32.704702 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:08:32.704702 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:08:32.704702 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:08:32.704702 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:08:32.704702 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:08:32.704702 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:08:32.704702 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:08:32.704702 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:08:32.704702 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:08:32.705991 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:08:32.705991 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:08:32.705991 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Aug 24 09:08:32.705991 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 320/646] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 24 09:08:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:08:32.705991 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249b77bd-8901-4e64-a631-2257e63e2984 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:32.756254 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:32.822376 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdc4707-1673-4860-a340-996d10ce3ba7 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:32.837605 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249b77bd-8901-4e64-a631-2257e63e2984 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:32.874098 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04586dd7-f370-4b4b-a9b5-7fa9abcbf8af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:32.874647 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04586dd7-f370-4b4b-a9b5-7fa9abcbf8af None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:32.874929 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04586dd7-f370-4b4b-a9b5-7fa9abcbf8af None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:32.875201 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04586dd7-f370-4b4b-a9b5-7fa9abcbf8af None admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:32.875967 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04586dd7-f370-4b4b-a9b5-7fa9abcbf8af None admin] RBAC: Adding query filter params (name=service) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:32.876273 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04586dd7-f370-4b4b-a9b5-7fa9abcbf8af None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:32.877646 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04586dd7-f370-4b4b-a9b5-7fa9abcbf8af None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:32.887644 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 327/647] 10.222.0.16 () {56 vars in 1103 bytes} [Wed Aug 24 09:08:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:32.888518 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04586dd7-f370-4b4b-a9b5-7fa9abcbf8af None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:32.890000 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7745b7b8-f376-4e30-acf4-c9967c9f4c90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:32.890646 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7745b7b8-f376-4e30-acf4-c9967c9f4c90 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:32.890923 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7745b7b8-f376-4e30-acf4-c9967c9f4c90 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:32.891193 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7745b7b8-f376-4e30-acf4-c9967c9f4c90 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:32.906360 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7745b7b8-f376-4e30-acf4-c9967c9f4c90 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:32.908248 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7745b7b8-f376-4e30-acf4-c9967c9f4c90 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:32.925457 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-7745b7b8-f376-4e30-acf4-c9967c9f4c90 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:08:32.925457 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:08:32.925457 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:08:32.925457 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:08:32.925457 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:08:32.925457 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:08:32.925457 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:08:32.925457 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:08:32.925457 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:08:32.925457 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:08:32.925457 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:08:32.925457 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:08:32.925457 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:08:32.925457 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:08:32.925457 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:08:32.925457 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:08:32.926699 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:08:32.926699 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:08:32.926699 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:08:32.926699 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:08:32.926699 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:08:32.926699 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:08:32.926699 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:08:32.926699 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Aug 24 09:08:32.926699 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:08:32.926699 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:08:32.926699 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:08:32.926699 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:08:32.926699 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:08:32.926699 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:08:32.926699 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:08:32.926699 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:08:32.926699 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:08:32.927834 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:08:32.927834 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:08:32.927834 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Aug 24 09:08:32.927834 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 321/648] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 24 09:08:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:08:32.928253 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7745b7b8-f376-4e30-acf4-c9967c9f4c90 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:32.986924 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:33.035483 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04586dd7-f370-4b4b-a9b5-7fa9abcbf8af None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:33.039742 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7745b7b8-f376-4e30-acf4-c9967c9f4c90 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:33.053239 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:33.079926 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9214c710-3cda-4cf2-9f25-8275be917a61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:33.081190 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9214c710-3cda-4cf2-9f25-8275be917a61 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:33.081648 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9214c710-3cda-4cf2-9f25-8275be917a61 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:33.082019 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10002d0-308e-431d-807d-4f73287ee258 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:33.082300 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9214c710-3cda-4cf2-9f25-8275be917a61 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:33.082641 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10002d0-308e-431d-807d-4f73287ee258 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:33.082903 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10002d0-308e-431d-807d-4f73287ee258 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:33.083170 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10002d0-308e-431d-807d-4f73287ee258 None admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:33.084082 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10002d0-308e-431d-807d-4f73287ee258 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:33.084388 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10002d0-308e-431d-807d-4f73287ee258 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:33.086302 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10002d0-308e-431d-807d-4f73287ee258 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:33.095185 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9214c710-3cda-4cf2-9f25-8275be917a61 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:33.097146 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9214c710-3cda-4cf2-9f25-8275be917a61 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:33.101418 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 328/649] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Aug 24 09:08:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:33.102331 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10002d0-308e-431d-807d-4f73287ee258 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:33.113362 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-9214c710-3cda-4cf2-9f25-8275be917a61 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:08:33.113362 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:08:33.113362 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:08:33.113362 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:08:33.113362 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:08:33.113362 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:08:33.113362 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:08:33.113362 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:08:33.113362 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:08:33.113362 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:08:33.113362 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:08:33.113362 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:08:33.113362 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:08:33.113362 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:08:33.113362 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:08:33.113362 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:08:33.114530 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:08:33.114530 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:08:33.114530 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:08:33.114530 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:08:33.114530 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:08:33.114530 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:08:33.114530 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:08:33.114530 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Aug 24 09:08:33.114530 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:08:33.114530 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:08:33.114530 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:08:33.114530 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:08:33.114530 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:08:33.114530 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:08:33.114530 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:08:33.114530 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:08:33.114530 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:08:33.115657 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:08:33.115657 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:08:33.115657 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Aug 24 09:08:33.117405 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 322/650] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 24 09:08:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:08:33.123432 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9214c710-3cda-4cf2-9f25-8275be917a61 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:33.222348 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10002d0-308e-431d-807d-4f73287ee258 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:33.240456 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9214c710-3cda-4cf2-9f25-8275be917a61 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:33.242354 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:33.259869 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3df52f6-8a21-4cc1-9280-b2be1777ef1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:33.263204 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3df52f6-8a21-4cc1-9280-b2be1777ef1a None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:33.263204 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3df52f6-8a21-4cc1-9280-b2be1777ef1a None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:33.263204 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3df52f6-8a21-4cc1-9280-b2be1777ef1a None admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:33.263204 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3df52f6-8a21-4cc1-9280-b2be1777ef1a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:33.263204 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3df52f6-8a21-4cc1-9280-b2be1777ef1a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:33.264142 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3df52f6-8a21-4cc1-9280-b2be1777ef1a None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:33.276078 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 329/651] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Aug 24 09:08:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:33.277620 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3df52f6-8a21-4cc1-9280-b2be1777ef1a None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:33.288292 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74d91b6-903e-47be-b8bd-08dbf527b48f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:33.288984 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74d91b6-903e-47be-b8bd-08dbf527b48f None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:33.289264 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74d91b6-903e-47be-b8bd-08dbf527b48f None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:33.289537 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74d91b6-903e-47be-b8bd-08dbf527b48f None admin] PATH_INFO: `/v3/users/glance` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:33.305109 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74d91b6-903e-47be-b8bd-08dbf527b48f None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:33.306860 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74d91b6-903e-47be-b8bd-08dbf527b48f None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:33.327787 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-d74d91b6-903e-47be-b8bd-08dbf527b48f None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 24 09:08:33.327787 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:08:33.327787 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:08:33.327787 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:08:33.327787 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:08:33.327787 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:08:33.327787 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:08:33.327787 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:08:33.327787 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:08:33.327787 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:08:33.327787 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:08:33.327787 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:08:33.327787 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 09:08:33.327787 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 09:08:33.327787 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 09:08:33.327787 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 09:08:33.328948 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:08:33.328948 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:08:33.328948 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 09:08:33.328948 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:08:33.328948 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 09:08:33.328948 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:08:33.328948 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:08:33.328948 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:08:33.328948 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:08:33.328948 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:08:33.328948 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:08:33.328948 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Aug 24 09:08:33.328948 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:08:33.328948 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:08:33.328948 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:08:33.328948 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:08:33.328948 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:08:33.328948 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:08:33.330184 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:08:33.330184 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:08:33.330184 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 09:08:33.330184 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 09:08:33.330184 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 09:08:33.330184 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 09:08:33.330184 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 09:08:33.330184 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 09:08:33.330184 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 24 09:08:33.330184 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Aug 24 09:08:33.330723 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:33.339641 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 323/652] 10.222.0.16 () {56 vars in 1099 bytes} [Wed Aug 24 09:08:33 2022] GET /identity/v3/users/glance => generated 84 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:08:33.339641 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74d91b6-903e-47be-b8bd-08dbf527b48f None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:33.404271 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3df52f6-8a21-4cc1-9280-b2be1777ef1a None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:33.448320 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5898c75b-31a9-4d2e-8f5e-b1e8cb97806d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:33.449184 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5898c75b-31a9-4d2e-8f5e-b1e8cb97806d None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:33.449464 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5898c75b-31a9-4d2e-8f5e-b1e8cb97806d None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:33.449733 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5898c75b-31a9-4d2e-8f5e-b1e8cb97806d None admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:33.450840 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5898c75b-31a9-4d2e-8f5e-b1e8cb97806d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:33.451152 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5898c75b-31a9-4d2e-8f5e-b1e8cb97806d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:33.452548 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5898c75b-31a9-4d2e-8f5e-b1e8cb97806d None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:33.467911 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74d91b6-903e-47be-b8bd-08dbf527b48f None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:33.473561 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 330/653] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Aug 24 09:08:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:33.474651 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5898c75b-31a9-4d2e-8f5e-b1e8cb97806d None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:33.505343 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd0d0bc-9c1d-4a5e-b122-aa92569bd82f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:33.506128 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd0d0bc-9c1d-4a5e-b122-aa92569bd82f None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:33.506519 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd0d0bc-9c1d-4a5e-b122-aa92569bd82f None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:33.506904 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd0d0bc-9c1d-4a5e-b122-aa92569bd82f None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:33.515268 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd0d0bc-9c1d-4a5e-b122-aa92569bd82f None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:33.516895 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd0d0bc-9c1d-4a5e-b122-aa92569bd82f None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:33.529321 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-0fd0d0bc-9c1d-4a5e-b122-aa92569bd82f None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 09:08:33.529321 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:08:33.529321 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:08:33.529321 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:08:33.529321 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:08:33.529321 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:08:33.529321 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:08:33.529321 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:08:33.529321 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:08:33.529321 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:08:33.529321 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:08:33.529321 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:08:33.529321 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 09:08:33.529321 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 09:08:33.529321 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 09:08:33.529321 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 09:08:33.530440 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:08:33.530440 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:08:33.530440 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 09:08:33.530440 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:08:33.530440 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 09:08:33.530440 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:08:33.530440 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:08:33.530440 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:08:33.530440 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:08:33.530440 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:08:33.530440 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:08:33.530440 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Aug 24 09:08:33.530440 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:08:33.530440 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:08:33.530440 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:08:33.530440 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:08:33.530440 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:08:33.530440 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:08:33.531628 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:08:33.531628 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:08:33.531628 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 09:08:33.531628 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 09:08:33.531628 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 09:08:33.531628 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 09:08:33.531628 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 09:08:33.531628 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 09:08:33.531628 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 09:08:33.531628 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Aug 24 09:08:33.534627 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 324/654] 10.222.0.16 () {56 vars in 1099 bytes} [Wed Aug 24 09:08:33 2022] GET /identity/v3/users/cinder => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:08:33.534627 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd0d0bc-9c1d-4a5e-b122-aa92569bd82f None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:33.572625 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:33.587399 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:33.595671 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5898c75b-31a9-4d2e-8f5e-b1e8cb97806d None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:33.636590 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c6b460-1172-46b1-a3f8-c56da62a0f26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:33.637206 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c6b460-1172-46b1-a3f8-c56da62a0f26 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:33.637486 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c6b460-1172-46b1-a3f8-c56da62a0f26 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:33.639099 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c6b460-1172-46b1-a3f8-c56da62a0f26 None admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:33.639353 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c6b460-1172-46b1-a3f8-c56da62a0f26 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:33.639701 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c6b460-1172-46b1-a3f8-c56da62a0f26 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:33.642184 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c6b460-1172-46b1-a3f8-c56da62a0f26 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:33.662649 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 331/655] 10.222.0.16 () {56 vars in 1137 bytes} [Wed Aug 24 09:08:33 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:33.663361 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c6b460-1172-46b1-a3f8-c56da62a0f26 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:33.679104 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd0d0bc-9c1d-4a5e-b122-aa92569bd82f None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:33.729897 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a960dfd-2fda-4357-b5be-22c8249a59ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:33.731232 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a960dfd-2fda-4357-b5be-22c8249a59ed None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:33.732000 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a960dfd-2fda-4357-b5be-22c8249a59ed None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:33.732682 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a960dfd-2fda-4357-b5be-22c8249a59ed None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:33.742782 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a960dfd-2fda-4357-b5be-22c8249a59ed None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:33.744433 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a960dfd-2fda-4357-b5be-22c8249a59ed None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:33.759679 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-8a960dfd-2fda-4357-b5be-22c8249a59ed None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 09:08:33.759679 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:08:33.759679 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:08:33.759679 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:08:33.759679 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:08:33.759679 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:08:33.759679 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:08:33.759679 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:08:33.759679 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:08:33.759679 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:08:33.759679 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:08:33.759679 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:08:33.759679 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 09:08:33.759679 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 09:08:33.759679 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 09:08:33.759679 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 09:08:33.761425 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:08:33.761425 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:08:33.761425 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 09:08:33.761425 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:08:33.761425 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 09:08:33.761425 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:08:33.761425 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:08:33.761425 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:08:33.761425 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:08:33.761425 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:08:33.761425 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:08:33.761425 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Aug 24 09:08:33.761425 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:08:33.761425 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:08:33.761425 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:08:33.761425 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:08:33.761425 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:08:33.761425 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:08:33.762603 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:08:33.762603 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:08:33.762603 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 09:08:33.762603 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 09:08:33.762603 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 09:08:33.762603 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 09:08:33.762603 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 09:08:33.762603 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 09:08:33.762603 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 09:08:33.762603 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Aug 24 09:08:33.764745 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 325/656] 10.222.0.16 () {56 vars in 1102 bytes} [Wed Aug 24 09:08:33 2022] GET /identity/v3/users/neutron => generated 85 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:08:33.765665 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a960dfd-2fda-4357-b5be-22c8249a59ed None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:33.776910 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c6b460-1172-46b1-a3f8-c56da62a0f26 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:33.816756 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2834f9-97ce-4b80-961d-f0cbfc6538ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:33.817636 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2834f9-97ce-4b80-961d-f0cbfc6538ae None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:33.818510 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2834f9-97ce-4b80-961d-f0cbfc6538ae None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:33.818938 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2834f9-97ce-4b80-961d-f0cbfc6538ae None admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:33.820283 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2834f9-97ce-4b80-961d-f0cbfc6538ae None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:33.820737 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2834f9-97ce-4b80-961d-f0cbfc6538ae None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:33.822869 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2834f9-97ce-4b80-961d-f0cbfc6538ae None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:33.852638 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 332/657] 10.222.0.16 () {56 vars in 1137 bytes} [Wed Aug 24 09:08:33 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:33.852816 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2834f9-97ce-4b80-961d-f0cbfc6538ae None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:33.867982 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:33.868482 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:33.892151 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a960dfd-2fda-4357-b5be-22c8249a59ed None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:33.930982 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0435e9bd-5e38-46bc-8999-97782585264b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:33.931562 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0435e9bd-5e38-46bc-8999-97782585264b None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:33.931980 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0435e9bd-5e38-46bc-8999-97782585264b None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:33.932272 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0435e9bd-5e38-46bc-8999-97782585264b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:33.987745 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2834f9-97ce-4b80-961d-f0cbfc6538ae None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:34.010686 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0435e9bd-5e38-46bc-8999-97782585264b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:34.013196 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0435e9bd-5e38-46bc-8999-97782585264b None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:34.025352 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5124a9-670f-4447-a624-808acc3a3770 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:34.025964 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5124a9-670f-4447-a624-808acc3a3770 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:34.026828 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5124a9-670f-4447-a624-808acc3a3770 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:34.027108 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5124a9-670f-4447-a624-808acc3a3770 None admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:34.027895 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5124a9-670f-4447-a624-808acc3a3770 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:34.028375 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5124a9-670f-4447-a624-808acc3a3770 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:34.029801 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5124a9-670f-4447-a624-808acc3a3770 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:34.047910 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 333/658] 10.222.0.16 () {56 vars in 1139 bytes} [Wed Aug 24 09:08:33 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:34.048824 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5124a9-670f-4447-a624-808acc3a3770 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:34.117836 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:34.120003 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:34.143744 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 326/659] 10.222.0.16 () {58 vars in 1282 bytes} [Wed Aug 24 09:08:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 09:08:34.143744 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0435e9bd-5e38-46bc-8999-97782585264b None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:34.169706 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5124a9-670f-4447-a624-808acc3a3770 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:34.206474 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7080001-982b-4b06-a335-2980d0389756 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:34.207283 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7080001-982b-4b06-a335-2980d0389756 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:34.207644 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7080001-982b-4b06-a335-2980d0389756 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:34.208028 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7080001-982b-4b06-a335-2980d0389756 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:34.274715 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0435e9bd-5e38-46bc-8999-97782585264b None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:34.289099 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7080001-982b-4b06-a335-2980d0389756 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:34.290640 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7080001-982b-4b06-a335-2980d0389756 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:34.314636 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2522f6db-ed0d-4f01-96b7-71a5befc4e5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:34.315654 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2522f6db-ed0d-4f01-96b7-71a5befc4e5b None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:34.316143 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2522f6db-ed0d-4f01-96b7-71a5befc4e5b None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:34.316594 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2522f6db-ed0d-4f01-96b7-71a5befc4e5b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:34.366081 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:34.372789 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:34.392894 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2522f6db-ed0d-4f01-96b7-71a5befc4e5b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:34.394668 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2522f6db-ed0d-4f01-96b7-71a5befc4e5b None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:34.409345 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 334/660] 10.222.0.16 () {58 vars in 1282 bytes} [Wed Aug 24 09:08:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 09:08:34.410290 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7080001-982b-4b06-a335-2980d0389756 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:34.506236 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 327/661] 10.222.0.16 () {58 vars in 1282 bytes} [Wed Aug 24 09:08:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 09:08:34.507450 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2522f6db-ed0d-4f01-96b7-71a5befc4e5b None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:34.521997 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7080001-982b-4b06-a335-2980d0389756 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:34.563604 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd60e28-0485-4129-b453-216ba846d0c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:34.564256 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd60e28-0485-4129-b453-216ba846d0c7 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:34.564532 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd60e28-0485-4129-b453-216ba846d0c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:34.565072 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd60e28-0485-4129-b453-216ba846d0c7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:34.576080 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd60e28-0485-4129-b453-216ba846d0c7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:34.578066 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd60e28-0485-4129-b453-216ba846d0c7 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:34.594808 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:34.599319 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-3dd60e28-0485-4129-b453-216ba846d0c7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:08:34.599319 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:08:34.599319 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:08:34.599319 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:08:34.599319 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:08:34.599319 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:08:34.599319 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:08:34.599319 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:08:34.599319 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:08:34.599319 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:08:34.599319 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:08:34.599319 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:08:34.599319 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:08:34.599319 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:08:34.599319 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:08:34.599319 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:08:34.600299 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:08:34.600299 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:08:34.600299 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:08:34.600299 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:08:34.600299 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:08:34.600299 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:08:34.600299 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:08:34.600299 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 24 09:08:34.600299 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:08:34.600299 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:08:34.600299 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:08:34.600299 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:08:34.600299 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:08:34.600299 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:08:34.600299 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:08:34.600299 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:08:34.600299 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:08:34.601200 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:08:34.601200 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:08:34.601200 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 24 09:08:34.602525 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 335/662] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 24 09:08:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:08:34.602717 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f02837-d146-4c5a-ac77-ae3a6ff81a1e None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:34.602997 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f02837-d146-4c5a-ac77-ae3a6ff81a1e None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:34.603280 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f02837-d146-4c5a-ac77-ae3a6ff81a1e None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:34.605152 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 336/663] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:08:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:08:34.609800 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f02837-d146-4c5a-ac77-ae3a6ff81a1e None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:34.619161 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2522f6db-ed0d-4f01-96b7-71a5befc4e5b None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:34.634131 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:34.655598 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa42b4dd-e414-4f38-a9f1-36afb5231327 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:34.656243 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa42b4dd-e414-4f38-a9f1-36afb5231327 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:34.656819 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa42b4dd-e414-4f38-a9f1-36afb5231327 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:34.656819 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa42b4dd-e414-4f38-a9f1-36afb5231327 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:34.668265 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa42b4dd-e414-4f38-a9f1-36afb5231327 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:34.670159 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa42b4dd-e414-4f38-a9f1-36afb5231327 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:34.686116 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-fa42b4dd-e414-4f38-a9f1-36afb5231327 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:08:34.686116 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:08:34.686116 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:08:34.686116 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:08:34.686116 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:08:34.686116 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:08:34.686116 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:08:34.686116 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:08:34.686116 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:08:34.686116 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:08:34.686116 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:08:34.686116 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:08:34.686116 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:08:34.686116 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:08:34.686116 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:08:34.686116 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:08:34.687101 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:08:34.687101 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:08:34.687101 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:08:34.687101 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:08:34.687101 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:08:34.687101 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:08:34.687101 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:08:34.687101 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Aug 24 09:08:34.687101 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:08:34.687101 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:08:34.687101 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:08:34.687101 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:08:34.687101 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:08:34.687101 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:08:34.687101 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:08:34.687101 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:08:34.687101 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:08:34.688035 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:08:34.688035 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:08:34.688035 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Aug 24 09:08:34.688035 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 328/664] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 24 09:08:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:08:34.688035 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa42b4dd-e414-4f38-a9f1-36afb5231327 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:34.727280 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f02837-d146-4c5a-ac77-ae3a6ff81a1e None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:34.763984 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b01bec8-3c74-4d0b-97b1-94487b3beb6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:34.764609 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b01bec8-3c74-4d0b-97b1-94487b3beb6e None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:34.764886 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b01bec8-3c74-4d0b-97b1-94487b3beb6e None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:34.765150 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b01bec8-3c74-4d0b-97b1-94487b3beb6e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:34.775514 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b01bec8-3c74-4d0b-97b1-94487b3beb6e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:34.777510 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b01bec8-3c74-4d0b-97b1-94487b3beb6e None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:34.792639 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-2b01bec8-3c74-4d0b-97b1-94487b3beb6e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:08:34.792639 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:08:34.792639 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:08:34.792639 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:08:34.792639 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:08:34.792639 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:08:34.792639 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:08:34.792639 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:08:34.792639 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:08:34.792639 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:08:34.792639 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:08:34.792639 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:08:34.792639 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:08:34.792639 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:08:34.792639 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:08:34.792639 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:08:34.793646 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:08:34.793646 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:08:34.793646 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:08:34.793646 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:08:34.793646 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:08:34.793646 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:08:34.793646 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:08:34.793646 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 24 09:08:34.793646 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:08:34.793646 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:08:34.793646 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:08:34.793646 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:08:34.793646 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:08:34.793646 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:08:34.793646 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:08:34.793646 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:08:34.793646 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:08:34.794528 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:08:34.794528 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:08:34.794528 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 24 09:08:34.794528 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 337/665] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 24 09:08:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:08:34.795016 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54893c50-bf31-46a9-a165-8b5e999ba1b5 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:34.795167 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54893c50-bf31-46a9-a165-8b5e999ba1b5 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:34.795420 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54893c50-bf31-46a9-a165-8b5e999ba1b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:34.807518 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa42b4dd-e414-4f38-a9f1-36afb5231327 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:34.812529 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-54893c50-bf31-46a9-a165-8b5e999ba1b5 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:08:34.813830 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-54893c50-bf31-46a9-a165-8b5e999ba1b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:34.849870 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64678f59-ac03-4055-8329-6f72ccac5820 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:34.850658 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64678f59-ac03-4055-8329-6f72ccac5820 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:34.851087 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64678f59-ac03-4055-8329-6f72ccac5820 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:34.851510 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64678f59-ac03-4055-8329-6f72ccac5820 None admin] PATH_INFO: `/v3/domains` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:34.852761 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64678f59-ac03-4055-8329-6f72ccac5820 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:34.853236 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64678f59-ac03-4055-8329-6f72ccac5820 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:34.855669 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64678f59-ac03-4055-8329-6f72ccac5820 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:34.857673 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:34.868979 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 329/666] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Aug 24 09:08:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:34.869901 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64678f59-ac03-4055-8329-6f72ccac5820 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:34.881343 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 338/667] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:08:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:08:34.882597 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0532c1b-6af6-4f32-a6d6-09f95d2992e7 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:34.882877 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0532c1b-6af6-4f32-a6d6-09f95d2992e7 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:34.883197 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0532c1b-6af6-4f32-a6d6-09f95d2992e7 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:34.884732 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 339/668] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:08:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:08:34.885297 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0532c1b-6af6-4f32-a6d6-09f95d2992e7 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:34.894137 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:34.981896 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64678f59-ac03-4055-8329-6f72ccac5820 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:34.994987 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0532c1b-6af6-4f32-a6d6-09f95d2992e7 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:35.018849 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c428be-9af5-426f-8b5e-7a45ee1282e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:35.019451 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c428be-9af5-426f-8b5e-7a45ee1282e8 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:35.019766 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c428be-9af5-426f-8b5e-7a45ee1282e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:35.020036 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c428be-9af5-426f-8b5e-7a45ee1282e8 None admin] PATH_INFO: `/v3/domains` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:35.020800 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c428be-9af5-426f-8b5e-7a45ee1282e8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:35.021084 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c428be-9af5-426f-8b5e-7a45ee1282e8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:35.022578 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c428be-9af5-426f-8b5e-7a45ee1282e8 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:35.029691 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840faebc-4887-4746-923f-5726306b5b41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:35.030293 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840faebc-4887-4746-923f-5726306b5b41 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:35.030587 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840faebc-4887-4746-923f-5726306b5b41 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:35.030852 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840faebc-4887-4746-923f-5726306b5b41 None admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:35.031493 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840faebc-4887-4746-923f-5726306b5b41 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:35.031826 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840faebc-4887-4746-923f-5726306b5b41 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:35.033200 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840faebc-4887-4746-923f-5726306b5b41 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:35.040756 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 330/669] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Aug 24 09:08:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:35.041658 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c428be-9af5-426f-8b5e-7a45ee1282e8 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:35.049608 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 340/670] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Aug 24 09:08:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:35.049608 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19114e1-c46c-49f2-9b1d-ab53b0a4dbc0 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:35.049608 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19114e1-c46c-49f2-9b1d-ab53b0a4dbc0 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:35.049608 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19114e1-c46c-49f2-9b1d-ab53b0a4dbc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:35.064609 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-c19114e1-c46c-49f2-9b1d-ab53b0a4dbc0 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:08:35.065730 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-c19114e1-c46c-49f2-9b1d-ab53b0a4dbc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:35.102859 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:35.128946 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:35.147125 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 341/671] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:08:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:08:35.147125 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19114e1-c46c-49f2-9b1d-ab53b0a4dbc0 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:35.163140 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c428be-9af5-426f-8b5e-7a45ee1282e8 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:35.200216 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a550e0b-ce3e-4cb2-85a2-3e3c5f3cd3de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:35.201039 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a550e0b-ce3e-4cb2-85a2-3e3c5f3cd3de None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:35.201394 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a550e0b-ce3e-4cb2-85a2-3e3c5f3cd3de None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:35.201749 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a550e0b-ce3e-4cb2-85a2-3e3c5f3cd3de None admin] PATH_INFO: `/v3/projects/service` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:35.209390 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a550e0b-ce3e-4cb2-85a2-3e3c5f3cd3de None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:35.210964 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a550e0b-ce3e-4cb2-85a2-3e3c5f3cd3de None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:35.223028 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-8a550e0b-ce3e-4cb2-85a2-3e3c5f3cd3de None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:08:35.223028 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:08:35.223028 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:08:35.223028 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:08:35.223028 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:08:35.223028 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:08:35.223028 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:08:35.223028 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:08:35.223028 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:08:35.223028 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:08:35.223028 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:08:35.223028 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:08:35.223028 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 09:08:35.223028 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 09:08:35.223028 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 09:08:35.224175 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 09:08:35.224175 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:08:35.224175 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:08:35.224175 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:08:35.224175 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:08:35.224175 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:08:35.224175 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:08:35.224175 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:08:35.224175 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Aug 24 09:08:35.224175 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:08:35.224175 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:08:35.224175 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:08:35.224175 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:08:35.224175 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:08:35.224175 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:08:35.224175 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:08:35.224175 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:08:35.225132 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 09:08:35.225132 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 09:08:35.225132 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 09:08:35.225132 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 09:08:35.225132 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 09:08:35.225132 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 09:08:35.225132 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:08:35.225132 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Aug 24 09:08:35.225132 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 331/672] 10.222.0.16 () {56 vars in 1111 bytes} [Wed Aug 24 09:08:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:08:35.225132 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a550e0b-ce3e-4cb2-85a2-3e3c5f3cd3de None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:35.250596 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19114e1-c46c-49f2-9b1d-ab53b0a4dbc0 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:35.288688 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf3749e-858d-49ed-801c-078a57edde39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:35.289764 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf3749e-858d-49ed-801c-078a57edde39 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:35.290470 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf3749e-858d-49ed-801c-078a57edde39 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:35.291193 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf3749e-858d-49ed-801c-078a57edde39 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:35.303615 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf3749e-858d-49ed-801c-078a57edde39 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:35.305096 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf3749e-858d-49ed-801c-078a57edde39 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:35.317478 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-abf3749e-858d-49ed-801c-078a57edde39 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:08:35.317478 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:08:35.317478 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:08:35.317478 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:08:35.317478 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:08:35.317478 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:08:35.317478 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:08:35.317478 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:08:35.317478 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:08:35.317478 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:08:35.317478 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:08:35.317478 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:08:35.317478 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 09:08:35.317478 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 09:08:35.317478 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 09:08:35.318585 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 09:08:35.318585 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:08:35.318585 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:08:35.318585 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:08:35.318585 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:08:35.318585 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:08:35.318585 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:08:35.318585 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:08:35.318585 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 24 09:08:35.318585 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:08:35.318585 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:08:35.318585 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:08:35.318585 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:08:35.318585 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:08:35.318585 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:08:35.318585 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:08:35.318585 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:08:35.319759 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 09:08:35.319759 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 09:08:35.319759 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 09:08:35.319759 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 09:08:35.319759 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 09:08:35.319759 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 09:08:35.319759 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:08:35.319759 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 24 09:08:35.324037 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 342/673] 10.222.0.16 () {56 vars in 1111 bytes} [Wed Aug 24 09:08:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:08:35.325115 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf3749e-858d-49ed-801c-078a57edde39 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:35.343670 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a550e0b-ce3e-4cb2-85a2-3e3c5f3cd3de None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:35.360709 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:35.381727 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:35.383119 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb4cf49-e72c-473c-a8a9-4b00e347b484 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:35.383774 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb4cf49-e72c-473c-a8a9-4b00e347b484 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:35.384085 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb4cf49-e72c-473c-a8a9-4b00e347b484 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:35.384328 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb4cf49-e72c-473c-a8a9-4b00e347b484 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:35.391181 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb4cf49-e72c-473c-a8a9-4b00e347b484 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:35.392674 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb4cf49-e72c-473c-a8a9-4b00e347b484 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:35.406892 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-bcb4cf49-e72c-473c-a8a9-4b00e347b484 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:08:35.406892 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:08:35.406892 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:08:35.406892 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:08:35.406892 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:08:35.406892 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:08:35.406892 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:08:35.406892 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:08:35.406892 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:08:35.406892 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:08:35.406892 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:08:35.406892 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:08:35.406892 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 09:08:35.406892 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 09:08:35.406892 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 09:08:35.407982 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 09:08:35.407982 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:08:35.407982 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:08:35.407982 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:08:35.407982 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:08:35.407982 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:08:35.407982 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:08:35.407982 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:08:35.407982 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Aug 24 09:08:35.407982 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:08:35.407982 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:08:35.407982 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:08:35.407982 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:08:35.407982 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:08:35.407982 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:08:35.407982 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:08:35.407982 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:08:35.409053 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 09:08:35.409053 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 09:08:35.409053 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 09:08:35.409053 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 09:08:35.409053 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 09:08:35.409053 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 09:08:35.409053 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:08:35.409053 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Aug 24 09:08:35.416287 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 332/674] 10.222.0.16 () {56 vars in 1111 bytes} [Wed Aug 24 09:08:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:08:35.417777 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb4cf49-e72c-473c-a8a9-4b00e347b484 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:35.430488 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf3749e-858d-49ed-801c-078a57edde39 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:35.464970 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56384787-3a77-4a87-bf20-f5f7a070ab74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:35.465668 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56384787-3a77-4a87-bf20-f5f7a070ab74 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:35.466017 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56384787-3a77-4a87-bf20-f5f7a070ab74 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:35.466361 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56384787-3a77-4a87-bf20-f5f7a070ab74 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:35.476351 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56384787-3a77-4a87-bf20-f5f7a070ab74 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:08:35.481564 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-56384787-3a77-4a87-bf20-f5f7a070ab74 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 09:08:35.481564 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:08:35.481564 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:08:35.481564 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:08:35.481564 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:08:35.481564 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:08:35.481564 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:08:35.481564 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:08:35.481564 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:08:35.481564 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:08:35.481564 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:08:35.481564 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:08:35.481564 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:08:35.481564 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:08:35.481564 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:08:35.481564 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:08:35.482669 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:08:35.482669 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:08:35.482669 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:08:35.482669 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:08:35.482669 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:08:35.482669 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:08:35.482669 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:08:35.482669 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:08:35.482669 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:08:35.482669 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:08:35.482669 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:08:35.482669 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 24 09:08:35.482669 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:08:35.482669 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:08:35.482669 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:08:35.482669 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:08:35.488423 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:08:35.488423 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:08:35.488423 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:08:35.488423 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:08:35.488423 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:08:35.488423 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:08:35.488423 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:08:35.488423 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:08:35.488423 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:08:35.488423 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:08:35.488423 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 09:08:35.488423 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 24 09:08:35.488423 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 343/675] 10.222.0.16 () {56 vars in 1102 bytes} [Wed Aug 24 09:08:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:08:35.488423 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56384787-3a77-4a87-bf20-f5f7a070ab74 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:35.517634 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb4cf49-e72c-473c-a8a9-4b00e347b484 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:35.551705 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cc3303-7e3d-4657-a8b9-f8c875eead6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:35.552306 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cc3303-7e3d-4657-a8b9-f8c875eead6b None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:35.552580 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cc3303-7e3d-4657-a8b9-f8c875eead6b None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:35.552852 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cc3303-7e3d-4657-a8b9-f8c875eead6b None admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:35.553538 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cc3303-7e3d-4657-a8b9-f8c875eead6b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:35.553842 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cc3303-7e3d-4657-a8b9-f8c875eead6b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:35.555253 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cc3303-7e3d-4657-a8b9-f8c875eead6b None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:35.567037 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 333/676] 10.222.0.16 () {56 vars in 1148 bytes} [Wed Aug 24 09:08:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:35.567937 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cc3303-7e3d-4657-a8b9-f8c875eead6b None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:35.584751 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56384787-3a77-4a87-bf20-f5f7a070ab74 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:35.598078 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:35.603414 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:35.618371 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f914e861-bd4c-4928-9d05-27726d25b334 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:35.618979 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f914e861-bd4c-4928-9d05-27726d25b334 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:35.619261 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f914e861-bd4c-4928-9d05-27726d25b334 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:35.619538 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f914e861-bd4c-4928-9d05-27726d25b334 None admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:35.620350 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f914e861-bd4c-4928-9d05-27726d25b334 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:35.620655 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f914e861-bd4c-4928-9d05-27726d25b334 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:35.622067 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f914e861-bd4c-4928-9d05-27726d25b334 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:35.634123 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 344/677] 10.222.0.16 () {56 vars in 1148 bytes} [Wed Aug 24 09:08:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:35.635210 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f914e861-bd4c-4928-9d05-27726d25b334 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:35.669531 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cc3303-7e3d-4657-a8b9-f8c875eead6b None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:35.702629 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4196378d-82f1-4c85-8ed7-4e6fa4068943 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:35.703250 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4196378d-82f1-4c85-8ed7-4e6fa4068943 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:35.703530 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4196378d-82f1-4c85-8ed7-4e6fa4068943 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:35.703834 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4196378d-82f1-4c85-8ed7-4e6fa4068943 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:35.715240 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4196378d-82f1-4c85-8ed7-4e6fa4068943 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:08:35.719962 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-4196378d-82f1-4c85-8ed7-4e6fa4068943 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:08:35.719962 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:08:35.719962 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:08:35.719962 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:08:35.719962 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:08:35.719962 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:08:35.719962 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:08:35.719962 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:08:35.719962 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:08:35.719962 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:08:35.719962 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:08:35.719962 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:08:35.719962 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:08:35.719962 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:08:35.719962 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:08:35.719962 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:08:35.721149 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:08:35.721149 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:08:35.721149 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:08:35.721149 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:08:35.721149 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:08:35.721149 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:08:35.721149 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:08:35.721149 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:08:35.721149 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:08:35.721149 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:08:35.721149 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:08:35.721149 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Aug 24 09:08:35.721149 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:08:35.721149 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:08:35.721149 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:08:35.721149 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:08:35.722271 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:08:35.722271 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:08:35.722271 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:08:35.722271 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:08:35.722271 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:08:35.722271 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:08:35.722271 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:08:35.722271 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:08:35.722271 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:08:35.722271 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:08:35.722271 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:08:35.722271 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Aug 24 09:08:35.722271 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 334/678] 10.222.0.16 () {56 vars in 1096 bytes} [Wed Aug 24 09:08:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:08:35.722271 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4196378d-82f1-4c85-8ed7-4e6fa4068943 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:35.738025 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f914e861-bd4c-4928-9d05-27726d25b334 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:35.770796 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4439c505-650f-4e3b-8eac-7651be5af43a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:35.771392 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4439c505-650f-4e3b-8eac-7651be5af43a None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:35.771702 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4439c505-650f-4e3b-8eac-7651be5af43a None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:35.771988 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4439c505-650f-4e3b-8eac-7651be5af43a None admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:35.772665 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4439c505-650f-4e3b-8eac-7651be5af43a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:35.772963 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4439c505-650f-4e3b-8eac-7651be5af43a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:35.774361 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4439c505-650f-4e3b-8eac-7651be5af43a None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:35.786473 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 345/679] 10.222.0.16 () {56 vars in 1148 bytes} [Wed Aug 24 09:08:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:35.787239 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4439c505-650f-4e3b-8eac-7651be5af43a None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:35.814991 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:35.816531 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:35.825820 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4196378d-82f1-4c85-8ed7-4e6fa4068943 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:35.855758 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22fd821-a389-413f-9d15-0fe7e317cc25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:35.856359 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22fd821-a389-413f-9d15-0fe7e317cc25 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:35.856641 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22fd821-a389-413f-9d15-0fe7e317cc25 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:35.856913 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22fd821-a389-413f-9d15-0fe7e317cc25 None admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:35.857577 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22fd821-a389-413f-9d15-0fe7e317cc25 None admin] RBAC: Adding query filter params (name=service) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:35.857881 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22fd821-a389-413f-9d15-0fe7e317cc25 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:35.859259 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22fd821-a389-413f-9d15-0fe7e317cc25 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:35.867254 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 335/680] 10.222.0.16 () {56 vars in 1103 bytes} [Wed Aug 24 09:08:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:35.868249 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22fd821-a389-413f-9d15-0fe7e317cc25 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:35.888823 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4439c505-650f-4e3b-8eac-7651be5af43a None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:35.922119 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dd49cb-c421-4bdd-a9b4-ded7e069682f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:35.922719 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dd49cb-c421-4bdd-a9b4-ded7e069682f None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:35.922997 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dd49cb-c421-4bdd-a9b4-ded7e069682f None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:35.923275 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dd49cb-c421-4bdd-a9b4-ded7e069682f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:35.924031 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25dd49cb-c421-4bdd-a9b4-ded7e069682f None admin] RBAC: Adding query filter params (role.id=67fd70a7d9794f7fb8faebffdec46dd2, scope.project.id=c467e88e2fb0491bacefcf6d2ce326d9, user.id=ea2666d430a646d386104addf1f42888) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:35.924340 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25dd49cb-c421-4bdd-a9b4-ded7e069682f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:35.925748 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25dd49cb-c421-4bdd-a9b4-ded7e069682f None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:35.931363 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 346/681] 10.222.0.16 () {56 vars in 1374 bytes} [Wed Aug 24 09:08:35 2022] GET /identity/v3/role_assignments?user.id=ea2666d430a646d386104addf1f42888&scope.project.id=c467e88e2fb0491bacefcf6d2ce326d9&role.id=67fd70a7d9794f7fb8faebffdec46dd2 => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:35.932326 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dd49cb-c421-4bdd-a9b4-ded7e069682f None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:35.992994 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22fd821-a389-413f-9d15-0fe7e317cc25 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:36.031922 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3947a33c-3e21-48ee-922f-b618fd3353e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:36.032813 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3947a33c-3e21-48ee-922f-b618fd3353e3 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:36.033388 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3947a33c-3e21-48ee-922f-b618fd3353e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:36.033824 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3947a33c-3e21-48ee-922f-b618fd3353e3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:36.034810 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3947a33c-3e21-48ee-922f-b618fd3353e3 None admin] RBAC: Adding query filter params (role.id=67fd70a7d9794f7fb8faebffdec46dd2, scope.project.id=c467e88e2fb0491bacefcf6d2ce326d9, user.id=05c32b83577c46488906fe409a790b24) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:36.035275 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3947a33c-3e21-48ee-922f-b618fd3353e3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:36.037333 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:36.037534 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3947a33c-3e21-48ee-922f-b618fd3353e3 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:36.043384 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dd49cb-c421-4bdd-a9b4-ded7e069682f None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:36.044832 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 336/682] 10.222.0.16 () {56 vars in 1374 bytes} [Wed Aug 24 09:08:35 2022] GET /identity/v3/role_assignments?user.id=05c32b83577c46488906fe409a790b24&scope.project.id=c467e88e2fb0491bacefcf6d2ce326d9&role.id=67fd70a7d9794f7fb8faebffdec46dd2 => generated 608 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:36.046887 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3947a33c-3e21-48ee-922f-b618fd3353e3 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:36.084464 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:36.084866 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd1f0d7-e8dc-46ce-bfef-635d2e0c219d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:36.085655 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd1f0d7-e8dc-46ce-bfef-635d2e0c219d None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:36.085892 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd1f0d7-e8dc-46ce-bfef-635d2e0c219d None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:36.086157 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd1f0d7-e8dc-46ce-bfef-635d2e0c219d None admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:36.086879 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd1f0d7-e8dc-46ce-bfef-635d2e0c219d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:36.087187 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd1f0d7-e8dc-46ce-bfef-635d2e0c219d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:36.089210 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd1f0d7-e8dc-46ce-bfef-635d2e0c219d None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:36.103808 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 347/683] 10.222.0.16 () {56 vars in 1099 bytes} [Wed Aug 24 09:08:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:36.105324 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd1f0d7-e8dc-46ce-bfef-635d2e0c219d None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:36.198677 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3947a33c-3e21-48ee-922f-b618fd3353e3 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:36.244544 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd1f0d7-e8dc-46ce-bfef-635d2e0c219d None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:36.249628 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9887a70f-132e-4411-8795-7f4a42f71edf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:36.249628 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9887a70f-132e-4411-8795-7f4a42f71edf None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:36.249842 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9887a70f-132e-4411-8795-7f4a42f71edf None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:36.250043 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9887a70f-132e-4411-8795-7f4a42f71edf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:36.250701 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9887a70f-132e-4411-8795-7f4a42f71edf None admin] RBAC: Adding query filter params (role.id=67fd70a7d9794f7fb8faebffdec46dd2, scope.project.id=c467e88e2fb0491bacefcf6d2ce326d9, user.id=8e5260e798404e8bb12b25ca8ea97052) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:36.251009 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9887a70f-132e-4411-8795-7f4a42f71edf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:36.252512 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9887a70f-132e-4411-8795-7f4a42f71edf None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:36.262192 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 337/684] 10.222.0.16 () {56 vars in 1374 bytes} [Wed Aug 24 09:08:36 2022] GET /identity/v3/role_assignments?user.id=8e5260e798404e8bb12b25ca8ea97052&scope.project.id=c467e88e2fb0491bacefcf6d2ce326d9&role.id=67fd70a7d9794f7fb8faebffdec46dd2 => generated 608 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:36.262897 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9887a70f-132e-4411-8795-7f4a42f71edf None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:36.296574 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c3fd4e-2add-4fe1-8501-897bfc563161 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:36.297270 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c3fd4e-2add-4fe1-8501-897bfc563161 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:36.297545 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c3fd4e-2add-4fe1-8501-897bfc563161 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:36.297859 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c3fd4e-2add-4fe1-8501-897bfc563161 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:36.312234 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c3fd4e-2add-4fe1-8501-897bfc563161 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:36.313706 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c3fd4e-2add-4fe1-8501-897bfc563161 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:36.335162 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-f9c3fd4e-2add-4fe1-8501-897bfc563161 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:08:36.335162 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:08:36.335162 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:08:36.335162 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:08:36.335162 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:08:36.335162 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:08:36.335162 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:08:36.335162 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:08:36.335162 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:08:36.335162 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:08:36.335162 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:08:36.335162 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:08:36.335162 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:08:36.335162 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:08:36.335162 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:08:36.335162 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:08:36.336210 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:08:36.336210 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:08:36.336210 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:08:36.336210 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:08:36.336210 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:08:36.336210 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:08:36.336210 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:08:36.336210 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 24 09:08:36.336210 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:08:36.336210 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:08:36.336210 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:08:36.336210 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:08:36.336210 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:08:36.336210 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:08:36.336210 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:08:36.336210 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:08:36.336210 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:08:36.337517 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:08:36.337517 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:08:36.337517 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 24 09:08:36.337517 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 348/685] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 24 09:08:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:08:36.345819 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c3fd4e-2add-4fe1-8501-897bfc563161 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:36.379781 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:36.423975 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:36.438780 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9887a70f-132e-4411-8795-7f4a42f71edf None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:36.485865 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b62c248-a040-4146-83bd-066da2084356 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:36.486513 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b62c248-a040-4146-83bd-066da2084356 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:36.486799 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b62c248-a040-4146-83bd-066da2084356 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:36.487094 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b62c248-a040-4146-83bd-066da2084356 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:36.501948 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b62c248-a040-4146-83bd-066da2084356 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:36.503978 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b62c248-a040-4146-83bd-066da2084356 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:36.528795 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-9b62c248-a040-4146-83bd-066da2084356 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:08:36.528795 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:08:36.528795 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:08:36.528795 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:08:36.528795 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:08:36.528795 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:08:36.528795 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:08:36.528795 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:08:36.528795 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:08:36.528795 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:08:36.528795 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:08:36.528795 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:08:36.528795 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:08:36.528795 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:08:36.528795 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:08:36.528795 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:08:36.529793 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:08:36.529793 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:08:36.529793 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:08:36.529793 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:08:36.529793 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:08:36.529793 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:08:36.529793 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:08:36.529793 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Aug 24 09:08:36.529793 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:08:36.529793 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:08:36.529793 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:08:36.529793 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:08:36.529793 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:08:36.529793 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:08:36.529793 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:08:36.529793 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:08:36.529793 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:08:36.530680 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:08:36.530680 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:08:36.530680 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Aug 24 09:08:36.530680 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 338/686] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 24 09:08:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:08:36.537656 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b62c248-a040-4146-83bd-066da2084356 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:36.547475 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c3fd4e-2add-4fe1-8501-897bfc563161 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:36.584643 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b59bab-bf2c-4e3d-bb0a-b85c4c600ba3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:36.585323 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b59bab-bf2c-4e3d-bb0a-b85c4c600ba3 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:36.585548 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b59bab-bf2c-4e3d-bb0a-b85c4c600ba3 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:36.585823 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b59bab-bf2c-4e3d-bb0a-b85c4c600ba3 None admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:36.586483 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b59bab-bf2c-4e3d-bb0a-b85c4c600ba3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:36.586789 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b59bab-bf2c-4e3d-bb0a-b85c4c600ba3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:36.588215 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b59bab-bf2c-4e3d-bb0a-b85c4c600ba3 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:36.602189 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 349/687] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Aug 24 09:08:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:36.607696 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b59bab-bf2c-4e3d-bb0a-b85c4c600ba3 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:36.654886 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b62c248-a040-4146-83bd-066da2084356 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:36.680182 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:36.694810 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fc8bfd-06f0-4a34-9811-1dbac39aee58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:36.695767 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fc8bfd-06f0-4a34-9811-1dbac39aee58 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:36.698040 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fc8bfd-06f0-4a34-9811-1dbac39aee58 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:36.698040 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fc8bfd-06f0-4a34-9811-1dbac39aee58 None admin] PATH_INFO: `/v3/domains` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:36.698040 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fc8bfd-06f0-4a34-9811-1dbac39aee58 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:36.698040 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fc8bfd-06f0-4a34-9811-1dbac39aee58 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:36.698903 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fc8bfd-06f0-4a34-9811-1dbac39aee58 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:36.701478 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:36.711738 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b59bab-bf2c-4e3d-bb0a-b85c4c600ba3 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:36.712276 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 339/688] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Aug 24 09:08:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:36.717634 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fc8bfd-06f0-4a34-9811-1dbac39aee58 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:36.748942 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837431a2-44dd-495b-b3e5-5ee224d2b521 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:36.749626 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837431a2-44dd-495b-b3e5-5ee224d2b521 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:36.749911 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837431a2-44dd-495b-b3e5-5ee224d2b521 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:36.750191 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837431a2-44dd-495b-b3e5-5ee224d2b521 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:36.757857 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837431a2-44dd-495b-b3e5-5ee224d2b521 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:36.759316 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837431a2-44dd-495b-b3e5-5ee224d2b521 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:36.771381 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-837431a2-44dd-495b-b3e5-5ee224d2b521 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 09:08:36.771381 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:08:36.771381 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:08:36.771381 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:08:36.771381 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:08:36.771381 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:08:36.771381 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:08:36.771381 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:08:36.771381 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:08:36.771381 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:08:36.771381 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:08:36.771381 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:08:36.771381 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 09:08:36.771381 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 09:08:36.771381 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 09:08:36.771381 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 09:08:36.772393 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:08:36.772393 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:08:36.772393 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 09:08:36.772393 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:08:36.772393 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 09:08:36.772393 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:08:36.772393 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:08:36.772393 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:08:36.772393 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:08:36.772393 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:08:36.772393 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:08:36.772393 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 24 09:08:36.772393 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:08:36.772393 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:08:36.772393 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:08:36.772393 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:08:36.772393 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:08:36.772393 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:08:36.773765 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:08:36.773765 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:08:36.773765 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 09:08:36.773765 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 09:08:36.773765 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 09:08:36.773765 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 09:08:36.773765 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 09:08:36.773765 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 09:08:36.773765 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 09:08:36.773765 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 24 09:08:36.773765 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 350/689] 10.222.0.16 () {56 vars in 1096 bytes} [Wed Aug 24 09:08:36 2022] GET /identity/v3/users/swift => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 09:08:36.778477 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837431a2-44dd-495b-b3e5-5ee224d2b521 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:36.825456 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fc8bfd-06f0-4a34-9811-1dbac39aee58 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:36.862654 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da976ec-5777-48c3-8067-d8d7b0069f47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:36.863180 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da976ec-5777-48c3-8067-d8d7b0069f47 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:36.863466 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da976ec-5777-48c3-8067-d8d7b0069f47 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:36.863774 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da976ec-5777-48c3-8067-d8d7b0069f47 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:36.872480 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da976ec-5777-48c3-8067-d8d7b0069f47 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:36.873922 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da976ec-5777-48c3-8067-d8d7b0069f47 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:36.884795 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837431a2-44dd-495b-b3e5-5ee224d2b521 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:36.886545 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-2da976ec-5777-48c3-8067-d8d7b0069f47 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 09:08:36.886545 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:08:36.886545 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:08:36.886545 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:08:36.886545 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:08:36.886545 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:08:36.886545 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:08:36.886545 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:08:36.886545 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:08:36.886545 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:08:36.886545 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:08:36.886545 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:08:36.886545 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 09:08:36.886545 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 09:08:36.886545 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 09:08:36.886545 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 09:08:36.887482 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:08:36.887482 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:08:36.887482 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 09:08:36.887482 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:08:36.887482 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 09:08:36.887482 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:08:36.887482 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:08:36.887482 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:08:36.887482 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:08:36.887482 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:08:36.887482 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:08:36.887482 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Aug 24 09:08:36.887482 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:08:36.887482 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:08:36.887482 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:08:36.887482 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:08:36.887482 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:08:36.887482 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:08:36.888922 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:08:36.888922 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:08:36.888922 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 09:08:36.888922 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 09:08:36.888922 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 09:08:36.888922 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 09:08:36.888922 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 09:08:36.888922 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 09:08:36.888922 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 09:08:36.888922 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Aug 24 09:08:36.888922 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 340/690] 10.222.0.16 () {56 vars in 1093 bytes} [Wed Aug 24 09:08:36 2022] GET /identity/v3/users/nova => generated 82 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 09:08:36.893596 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da976ec-5777-48c3-8067-d8d7b0069f47 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:36.915504 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:36.940364 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695bc03e-dc0c-42c0-acc9-a76f6986fa1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:36.940935 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695bc03e-dc0c-42c0-acc9-a76f6986fa1d None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:36.941235 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695bc03e-dc0c-42c0-acc9-a76f6986fa1d None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:36.941500 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695bc03e-dc0c-42c0-acc9-a76f6986fa1d None admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:36.942243 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695bc03e-dc0c-42c0-acc9-a76f6986fa1d None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:36.942552 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695bc03e-dc0c-42c0-acc9-a76f6986fa1d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:36.944049 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695bc03e-dc0c-42c0-acc9-a76f6986fa1d None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:36.959971 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:36.990388 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 351/691] 10.222.0.16 () {56 vars in 1135 bytes} [Wed Aug 24 09:08:36 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:08:36.996189 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695bc03e-dc0c-42c0-acc9-a76f6986fa1d None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:37.003048 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da976ec-5777-48c3-8067-d8d7b0069f47 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:37.039629 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14c1829-84b0-4b58-8f4e-d8b550c94ae6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:37.040462 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14c1829-84b0-4b58-8f4e-d8b550c94ae6 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:37.040886 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14c1829-84b0-4b58-8f4e-d8b550c94ae6 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:37.041303 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14c1829-84b0-4b58-8f4e-d8b550c94ae6 None admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:37.042766 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14c1829-84b0-4b58-8f4e-d8b550c94ae6 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:37.043273 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14c1829-84b0-4b58-8f4e-d8b550c94ae6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:37.045336 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14c1829-84b0-4b58-8f4e-d8b550c94ae6 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:37.065173 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 341/692] 10.222.0.16 () {56 vars in 1133 bytes} [Wed Aug 24 09:08:36 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:08:37.070798 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14c1829-84b0-4b58-8f4e-d8b550c94ae6 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:37.156914 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695bc03e-dc0c-42c0-acc9-a76f6986fa1d None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:37.186662 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14c1829-84b0-4b58-8f4e-d8b550c94ae6 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:37.192494 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea6a25d-dae0-43d6-8bd5-60a9a2f54335 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:37.193540 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea6a25d-dae0-43d6-8bd5-60a9a2f54335 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:37.193540 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea6a25d-dae0-43d6-8bd5-60a9a2f54335 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:37.193713 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea6a25d-dae0-43d6-8bd5-60a9a2f54335 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:37.217146 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:37.253929 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:37.265252 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79793db1-87fd-4c4b-9223-510a248e4af7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:37.265875 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79793db1-87fd-4c4b-9223-510a248e4af7 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:37.266156 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79793db1-87fd-4c4b-9223-510a248e4af7 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:37.266431 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79793db1-87fd-4c4b-9223-510a248e4af7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:37.271002 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea6a25d-dae0-43d6-8bd5-60a9a2f54335 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:37.272496 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea6a25d-dae0-43d6-8bd5-60a9a2f54335 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:37.390518 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 352/693] 10.222.0.16 () {58 vars in 1282 bytes} [Wed Aug 24 09:08:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 09:08:37.396328 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea6a25d-dae0-43d6-8bd5-60a9a2f54335 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:37.439532 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79793db1-87fd-4c4b-9223-510a248e4af7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:37.441856 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79793db1-87fd-4c4b-9223-510a248e4af7 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:37.555363 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:37.570780 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea6a25d-dae0-43d6-8bd5-60a9a2f54335 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:37.600951 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 342/694] 10.222.0.16 () {58 vars in 1282 bytes} [Wed Aug 24 09:08:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 530 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 09:08:37.607254 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79793db1-87fd-4c4b-9223-510a248e4af7 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:37.614669 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a9a4a8-4197-4e86-b98c-31f917ac86dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:37.615411 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a9a4a8-4197-4e86-b98c-31f917ac86dc None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:37.615818 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a9a4a8-4197-4e86-b98c-31f917ac86dc None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:37.616098 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a9a4a8-4197-4e86-b98c-31f917ac86dc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:37.628077 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a9a4a8-4197-4e86-b98c-31f917ac86dc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:37.629566 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a9a4a8-4197-4e86-b98c-31f917ac86dc None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:37.644728 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:37.646472 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-98a9a4a8-4197-4e86-b98c-31f917ac86dc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:08:37.646472 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:08:37.646472 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:08:37.646472 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:08:37.646472 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:08:37.646472 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:08:37.646472 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:08:37.646472 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:08:37.646472 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:08:37.646472 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:08:37.646472 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:08:37.646472 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:08:37.646472 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:08:37.646472 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:08:37.646472 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:08:37.646472 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:08:37.647447 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:08:37.647447 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:08:37.647447 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:08:37.647447 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:08:37.647447 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:08:37.647447 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:08:37.647447 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:08:37.647447 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 24 09:08:37.647447 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:08:37.647447 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:08:37.647447 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:08:37.647447 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:08:37.647447 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:08:37.647447 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:08:37.647447 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:08:37.647447 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:08:37.647447 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:08:37.648431 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:08:37.648431 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:08:37.648431 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 24 09:08:37.648431 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 353/695] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 24 09:08:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 09:08:37.654884 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a9a4a8-4197-4e86-b98c-31f917ac86dc None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:37.782042 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79793db1-87fd-4c4b-9223-510a248e4af7 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:37.794871 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a9a4a8-4197-4e86-b98c-31f917ac86dc None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:37.832042 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81e2ce8-4534-4ccf-b423-c47833c6e2d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:37.832658 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81e2ce8-4534-4ccf-b423-c47833c6e2d6 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:37.832943 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81e2ce8-4534-4ccf-b423-c47833c6e2d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:37.833219 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81e2ce8-4534-4ccf-b423-c47833c6e2d6 None admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:37.833870 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81e2ce8-4534-4ccf-b423-c47833c6e2d6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:37.834178 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81e2ce8-4534-4ccf-b423-c47833c6e2d6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:37.835552 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81e2ce8-4534-4ccf-b423-c47833c6e2d6 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:37.845276 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63cd81c-fbcd-4579-bae2-a4045012eabb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:37.845552 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:37.845982 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63cd81c-fbcd-4579-bae2-a4045012eabb None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:37.846265 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63cd81c-fbcd-4579-bae2-a4045012eabb None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:37.846647 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63cd81c-fbcd-4579-bae2-a4045012eabb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:37.849712 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 354/696] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Aug 24 09:08:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:08:37.854986 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81e2ce8-4534-4ccf-b423-c47833c6e2d6 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:37.872554 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63cd81c-fbcd-4579-bae2-a4045012eabb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:37.874236 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63cd81c-fbcd-4579-bae2-a4045012eabb None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:37.898332 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-a63cd81c-fbcd-4579-bae2-a4045012eabb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:08:37.898332 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:08:37.898332 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:08:37.898332 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:08:37.898332 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:08:37.898332 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:08:37.898332 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:08:37.898332 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:08:37.898332 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:08:37.898332 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:08:37.898332 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:08:37.898332 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:08:37.898332 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:08:37.898332 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:08:37.898332 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:08:37.898332 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:08:37.900581 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:08:37.900581 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:08:37.900581 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:08:37.900581 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:08:37.900581 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:08:37.900581 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:08:37.900581 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:08:37.900581 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Aug 24 09:08:37.900581 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:08:37.900581 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:08:37.900581 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:08:37.900581 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:08:37.900581 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:08:37.900581 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:08:37.900581 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:08:37.900581 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:08:37.900581 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:08:37.902690 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:08:37.902690 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:08:37.902690 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Aug 24 09:08:37.902690 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 343/697] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 24 09:08:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:08:37.908595 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63cd81c-fbcd-4579-bae2-a4045012eabb None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:37.962850 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81e2ce8-4534-4ccf-b423-c47833c6e2d6 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:37.999246 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cde82f1-ba45-4298-94b3-f2cf7ccf020b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:38.000049 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cde82f1-ba45-4298-94b3-f2cf7ccf020b None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:38.000328 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cde82f1-ba45-4298-94b3-f2cf7ccf020b None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:38.000607 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cde82f1-ba45-4298-94b3-f2cf7ccf020b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:38.008166 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cde82f1-ba45-4298-94b3-f2cf7ccf020b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:38.009637 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cde82f1-ba45-4298-94b3-f2cf7ccf020b None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:38.022255 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-6cde82f1-ba45-4298-94b3-f2cf7ccf020b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:08:38.022255 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:08:38.022255 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:08:38.022255 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:08:38.022255 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:08:38.022255 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:08:38.022255 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:08:38.022255 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:08:38.022255 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:08:38.022255 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:08:38.022255 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:08:38.022255 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:08:38.022255 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 09:08:38.022255 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 09:08:38.022255 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 09:08:38.024399 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 09:08:38.024399 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:08:38.024399 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:08:38.024399 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:08:38.024399 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:08:38.024399 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:08:38.024399 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:08:38.024399 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:08:38.024399 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 24 09:08:38.024399 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:08:38.024399 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:08:38.024399 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:08:38.024399 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:08:38.024399 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:08:38.024399 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:08:38.024399 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:08:38.024399 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:08:38.026587 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 09:08:38.026587 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 09:08:38.026587 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 09:08:38.026587 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 09:08:38.026587 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 09:08:38.026587 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 09:08:38.026587 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:08:38.026587 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 24 09:08:38.027753 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 355/698] 10.222.0.16 () {56 vars in 1111 bytes} [Wed Aug 24 09:08:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:08:38.029467 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cde82f1-ba45-4298-94b3-f2cf7ccf020b None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:38.102316 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:38.111966 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:38.139461 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cde82f1-ba45-4298-94b3-f2cf7ccf020b None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:38.176703 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32862238-eecd-4674-a1f1-a77fefc520ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:38.177324 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32862238-eecd-4674-a1f1-a77fefc520ed None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:38.177607 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32862238-eecd-4674-a1f1-a77fefc520ed None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:38.177881 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32862238-eecd-4674-a1f1-a77fefc520ed None admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:38.178582 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32862238-eecd-4674-a1f1-a77fefc520ed None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:38.178888 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32862238-eecd-4674-a1f1-a77fefc520ed None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:38.180501 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32862238-eecd-4674-a1f1-a77fefc520ed None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:38.198882 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 356/699] 10.222.0.16 () {56 vars in 1148 bytes} [Wed Aug 24 09:08:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:08:38.200240 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32862238-eecd-4674-a1f1-a77fefc520ed None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:38.205832 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63cd81c-fbcd-4579-bae2-a4045012eabb None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:38.259570 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3731476-d21b-432a-8687-1f8327435409 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:38.260749 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3731476-d21b-432a-8687-1f8327435409 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:38.261037 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3731476-d21b-432a-8687-1f8327435409 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:38.261319 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3731476-d21b-432a-8687-1f8327435409 None admin] PATH_INFO: `/v3/domains` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:38.261978 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3731476-d21b-432a-8687-1f8327435409 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:38.262289 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3731476-d21b-432a-8687-1f8327435409 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:38.263754 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3731476-d21b-432a-8687-1f8327435409 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:38.278531 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 344/700] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Aug 24 09:08:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 370 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:08:38.283171 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede3a6c2-d07c-44d4-9003-3276610013cb None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:38.283171 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede3a6c2-d07c-44d4-9003-3276610013cb None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:38.283171 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede3a6c2-d07c-44d4-9003-3276610013cb None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:38.283171 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 345/701] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:08:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:08:38.284427 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede3a6c2-d07c-44d4-9003-3276610013cb None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:38.347195 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32862238-eecd-4674-a1f1-a77fefc520ed None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:38.368489 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:38.384387 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ba48de-79ba-4a39-8014-9fe4043ed1a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:38.396061 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ba48de-79ba-4a39-8014-9fe4043ed1a5 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:38.396061 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ba48de-79ba-4a39-8014-9fe4043ed1a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:38.396061 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ba48de-79ba-4a39-8014-9fe4043ed1a5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:38.396061 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ba48de-79ba-4a39-8014-9fe4043ed1a5 None admin] RBAC: Adding query filter params (role.id=67fd70a7d9794f7fb8faebffdec46dd2, scope.project.id=c467e88e2fb0491bacefcf6d2ce326d9, user.id=fdd5f0d67fe54e9fae6f4af644988d93) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:38.396061 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ba48de-79ba-4a39-8014-9fe4043ed1a5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:38.396061 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ba48de-79ba-4a39-8014-9fe4043ed1a5 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:38.396567 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 357/702] 10.222.0.16 () {56 vars in 1374 bytes} [Wed Aug 24 09:08:38 2022] GET /identity/v3/role_assignments?user.id=fdd5f0d67fe54e9fae6f4af644988d93&scope.project.id=c467e88e2fb0491bacefcf6d2ce326d9&role.id=67fd70a7d9794f7fb8faebffdec46dd2 => generated 608 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:38.396567 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ddbc82-fca9-4741-8ccc-098849a71f86 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:38.396567 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ddbc82-fca9-4741-8ccc-098849a71f86 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:38.397026 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ddbc82-fca9-4741-8ccc-098849a71f86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:38.406582 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede3a6c2-d07c-44d4-9003-3276610013cb None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:38.417743 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-09ddbc82-fca9-4741-8ccc-098849a71f86 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:08:38.419615 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-09ddbc82-fca9-4741-8ccc-098849a71f86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:38.437760 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:38.447140 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a417cfd6-e35f-4ae6-932f-1ef7da0f7979 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:38.448118 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a417cfd6-e35f-4ae6-932f-1ef7da0f7979 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:38.448503 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a417cfd6-e35f-4ae6-932f-1ef7da0f7979 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:38.448886 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a417cfd6-e35f-4ae6-932f-1ef7da0f7979 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:38.457450 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a417cfd6-e35f-4ae6-932f-1ef7da0f7979 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:38.459074 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a417cfd6-e35f-4ae6-932f-1ef7da0f7979 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:38.471617 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-a417cfd6-e35f-4ae6-932f-1ef7da0f7979 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:08:38.471617 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:08:38.471617 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:08:38.471617 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:08:38.471617 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:08:38.471617 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:08:38.471617 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:08:38.471617 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:08:38.471617 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:08:38.471617 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:08:38.471617 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:08:38.471617 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:08:38.471617 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 09:08:38.471617 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 09:08:38.471617 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 09:08:38.472780 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 09:08:38.472780 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:08:38.472780 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:08:38.472780 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:08:38.472780 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:08:38.472780 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:08:38.472780 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:08:38.472780 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:08:38.472780 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Aug 24 09:08:38.472780 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:08:38.472780 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:08:38.472780 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:08:38.472780 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:08:38.472780 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:08:38.472780 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:08:38.472780 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:08:38.472780 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:08:38.473962 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 09:08:38.473962 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 09:08:38.473962 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 09:08:38.473962 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 09:08:38.473962 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 09:08:38.473962 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 09:08:38.473962 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:08:38.473962 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Aug 24 09:08:38.480621 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 346/703] 10.222.0.16 () {56 vars in 1111 bytes} [Wed Aug 24 09:08:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 09:08:38.480621 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6835111f-d4e1-4052-97e1-6d2938bd90ee None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:38.480621 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6835111f-d4e1-4052-97e1-6d2938bd90ee None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:38.480621 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6835111f-d4e1-4052-97e1-6d2938bd90ee None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:38.480621 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 347/704] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:08:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:08:38.482233 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6835111f-d4e1-4052-97e1-6d2938bd90ee None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:38.511773 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 358/705] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:08:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:08:38.513478 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed59b962-5a7c-49e3-9bcc-113df500d5f1 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:38.513949 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed59b962-5a7c-49e3-9bcc-113df500d5f1 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:38.514426 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed59b962-5a7c-49e3-9bcc-113df500d5f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:38.552643 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-ed59b962-5a7c-49e3-9bcc-113df500d5f1 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:08:38.553846 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-ed59b962-5a7c-49e3-9bcc-113df500d5f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:38.623291 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6835111f-d4e1-4052-97e1-6d2938bd90ee None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:38.637398 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 359/706] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:08:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:08:38.666315 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066338db-3717-413a-b6e7-7b9d70e4db86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:38.667778 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066338db-3717-413a-b6e7-7b9d70e4db86 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:38.668215 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066338db-3717-413a-b6e7-7b9d70e4db86 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:38.668642 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066338db-3717-413a-b6e7-7b9d70e4db86 None admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:38.669501 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066338db-3717-413a-b6e7-7b9d70e4db86 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:38.669981 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066338db-3717-413a-b6e7-7b9d70e4db86 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:38.671552 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066338db-3717-413a-b6e7-7b9d70e4db86 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:38.685748 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 348/707] 10.222.0.16 () {56 vars in 1148 bytes} [Wed Aug 24 09:08:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:38.691346 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed59b962-5a7c-49e3-9bcc-113df500d5f1 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:38.748634 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:38.806338 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed59b962-5a7c-49e3-9bcc-113df500d5f1 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:38.824708 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066338db-3717-413a-b6e7-7b9d70e4db86 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:38.853411 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e219ec8e-08fd-4844-bfa7-ae6fcacede35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:38.854587 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e219ec8e-08fd-4844-bfa7-ae6fcacede35 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:38.854876 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e219ec8e-08fd-4844-bfa7-ae6fcacede35 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:38.855147 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e219ec8e-08fd-4844-bfa7-ae6fcacede35 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:38.855916 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e219ec8e-08fd-4844-bfa7-ae6fcacede35 None admin] RBAC: Adding query filter params (role.id=15f634d4a2d247ddab90c458e5e1c141, scope.project.id=c467e88e2fb0491bacefcf6d2ce326d9, user.id=1f055f7dc51d43bda9b315ca67f11d58) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:38.856227 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e219ec8e-08fd-4844-bfa7-ae6fcacede35 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:38.857791 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e219ec8e-08fd-4844-bfa7-ae6fcacede35 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:38.863706 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 360/708] 10.222.0.16 () {56 vars in 1374 bytes} [Wed Aug 24 09:08:38 2022] GET /identity/v3/role_assignments?user.id=1f055f7dc51d43bda9b315ca67f11d58&scope.project.id=c467e88e2fb0491bacefcf6d2ce326d9&role.id=15f634d4a2d247ddab90c458e5e1c141 => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:38.883658 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:38.931405 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e219ec8e-08fd-4844-bfa7-ae6fcacede35 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:38.951628 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066338db-3717-413a-b6e7-7b9d70e4db86 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:39.003937 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e683abd-2512-441a-9994-c12deef5fd30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:39.004606 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e683abd-2512-441a-9994-c12deef5fd30 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:39.004883 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e683abd-2512-441a-9994-c12deef5fd30 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:39.005156 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e683abd-2512-441a-9994-c12deef5fd30 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:39.005760 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e683abd-2512-441a-9994-c12deef5fd30 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:39.007176 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e683abd-2512-441a-9994-c12deef5fd30 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:39.037676 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-1e683abd-2512-441a-9994-c12deef5fd30 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 09:08:39.037676 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:08:39.037676 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 09:08:39.037676 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 09:08:39.037676 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 09:08:39.037676 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 09:08:39.037676 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 09:08:39.037676 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:08:39.037676 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 09:08:39.037676 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Aug 24 09:08:39.037676 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 09:08:39.037676 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Aug 24 09:08:39.037676 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:08:39.037676 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:08:39.037676 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:08:39.037676 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:08:39.037676 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:08:39.038803 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:08:39.038803 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:08:39.038803 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:08:39.038803 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:08:39.038803 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:08:39.038803 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:08:39.038803 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 09:08:39.038803 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 09:08:39.038803 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 09:08:39.038803 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 09:08:39.038803 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:08:39.038803 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:08:39.038803 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:08:39.038803 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:08:39.038803 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:08:39.038803 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:08:39.040379 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:08:39.040379 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Aug 24 09:08:39.040379 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:08:39.040379 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:08:39.040379 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:08:39.040379 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:08:39.040379 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:08:39.040379 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:08:39.040379 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:08:39.040379 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:08:39.040379 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 09:08:39.040379 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:08:39.040379 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 09:08:39.040379 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Aug 24 09:08:39.040379 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 349/709] 10.222.0.16 () {56 vars in 1129 bytes} [Wed Aug 24 09:08:38 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 09:08:39.045384 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e683abd-2512-441a-9994-c12deef5fd30 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:39.119058 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e219ec8e-08fd-4844-bfa7-ae6fcacede35 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:39.130069 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:39.157431 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d51e57c-872d-4ee2-bccc-bc0ad53aa395 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:39.158412 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d51e57c-872d-4ee2-bccc-bc0ad53aa395 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:39.158923 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d51e57c-872d-4ee2-bccc-bc0ad53aa395 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:39.159399 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d51e57c-872d-4ee2-bccc-bc0ad53aa395 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:39.170670 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d51e57c-872d-4ee2-bccc-bc0ad53aa395 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:39.173450 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d51e57c-872d-4ee2-bccc-bc0ad53aa395 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:39.185607 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e683abd-2512-441a-9994-c12deef5fd30 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:39.189404 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-4d51e57c-872d-4ee2-bccc-bc0ad53aa395 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:08:39.189404 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:08:39.189404 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:08:39.189404 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:08:39.189404 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:08:39.189404 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:08:39.189404 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:08:39.189404 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:08:39.189404 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:08:39.189404 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:08:39.189404 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:08:39.189404 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:08:39.189404 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:08:39.189404 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:08:39.189404 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:08:39.189404 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:08:39.190354 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:08:39.190354 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:08:39.190354 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:08:39.190354 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:08:39.190354 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:08:39.190354 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:08:39.190354 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:08:39.190354 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 24 09:08:39.190354 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:08:39.190354 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:08:39.190354 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:08:39.190354 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:08:39.190354 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:08:39.190354 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:08:39.190354 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:08:39.190354 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:08:39.190354 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:08:39.191284 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:08:39.191284 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:08:39.191284 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 24 09:08:39.193432 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 361/710] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 24 09:08:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 09:08:39.194984 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9b0d8c-3178-41fc-8c54-29e44f2e2692 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:39.195480 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9b0d8c-3178-41fc-8c54-29e44f2e2692 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:39.195980 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9b0d8c-3178-41fc-8c54-29e44f2e2692 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:39.199967 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:39.202316 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 362/711] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:08:39 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:08:39.203120 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9b0d8c-3178-41fc-8c54-29e44f2e2692 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:39.225989 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e73d84-ce8b-4047-b0d5-21cb31b2da8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:39.226608 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e73d84-ce8b-4047-b0d5-21cb31b2da8d None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:39.226885 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e73d84-ce8b-4047-b0d5-21cb31b2da8d None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:39.227154 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e73d84-ce8b-4047-b0d5-21cb31b2da8d None admin] PATH_INFO: `/v3/services` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:39.228237 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e73d84-ce8b-4047-b0d5-21cb31b2da8d None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:39.231149 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e73d84-ce8b-4047-b0d5-21cb31b2da8d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:39.231149 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e73d84-ce8b-4047-b0d5-21cb31b2da8d None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:39.236108 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 350/712] 10.222.0.16 () {56 vars in 1124 bytes} [Wed Aug 24 09:08:39 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:39.237435 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55baa245-2f53-4c57-8ed3-27303f4a1af3 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:39.237720 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55baa245-2f53-4c57-8ed3-27303f4a1af3 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:39.237995 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55baa245-2f53-4c57-8ed3-27303f4a1af3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:39.257402 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-55baa245-2f53-4c57-8ed3-27303f4a1af3 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:08:39.258631 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-55baa245-2f53-4c57-8ed3-27303f4a1af3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:39.335917 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 351/713] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:08:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:08:39.336837 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55baa245-2f53-4c57-8ed3-27303f4a1af3 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:39.340258 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9b0d8c-3178-41fc-8c54-29e44f2e2692 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:39.404489 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df20c7fd-f25e-4ded-9170-af2bf3f201d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:39.405106 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df20c7fd-f25e-4ded-9170-af2bf3f201d8 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:39.405380 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df20c7fd-f25e-4ded-9170-af2bf3f201d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:39.405650 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df20c7fd-f25e-4ded-9170-af2bf3f201d8 None admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:39.406297 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df20c7fd-f25e-4ded-9170-af2bf3f201d8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:39.406596 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df20c7fd-f25e-4ded-9170-af2bf3f201d8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:39.408131 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df20c7fd-f25e-4ded-9170-af2bf3f201d8 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:39.430369 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 363/714] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Aug 24 09:08:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:39.434855 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df20c7fd-f25e-4ded-9170-af2bf3f201d8 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:39.459931 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:39.545123 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55baa245-2f53-4c57-8ed3-27303f4a1af3 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:39.551564 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:39.577989 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df20c7fd-f25e-4ded-9170-af2bf3f201d8 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:39.616025 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9baae798-e58a-4070-b7a3-3843cae9e51a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:39.616639 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baae798-e58a-4070-b7a3-3843cae9e51a None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:39.616920 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baae798-e58a-4070-b7a3-3843cae9e51a None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:39.617189 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baae798-e58a-4070-b7a3-3843cae9e51a None admin] PATH_INFO: `/v3/services` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:39.617825 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9baae798-e58a-4070-b7a3-3843cae9e51a None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:39.618127 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9baae798-e58a-4070-b7a3-3843cae9e51a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:39.619497 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9baae798-e58a-4070-b7a3-3843cae9e51a None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:39.631761 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 352/715] 10.222.0.16 () {56 vars in 1124 bytes} [Wed Aug 24 09:08:39 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:39.652481 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f543db52-941c-4728-94ce-9dc380d1109d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:39.653133 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f543db52-941c-4728-94ce-9dc380d1109d None admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:39.653415 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f543db52-941c-4728-94ce-9dc380d1109d None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:39.653689 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f543db52-941c-4728-94ce-9dc380d1109d None admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:39.654365 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f543db52-941c-4728-94ce-9dc380d1109d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:39.655889 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f543db52-941c-4728-94ce-9dc380d1109d None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:39.704168 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9baae798-e58a-4070-b7a3-3843cae9e51a None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:39.756040 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 364/716] 10.222.0.16 () {60 vars in 1132 bytes} [Wed Aug 24 09:08:39 2022] POST /identity/v3/users => generated 254 bytes in 322 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 09:08:39.828020 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9baae798-e58a-4070-b7a3-3843cae9e51a None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:39.873450 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98a5a3f-da5b-4585-b153-7a8a05f51b61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:39.874493 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98a5a3f-da5b-4585-b153-7a8a05f51b61 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:39.874982 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98a5a3f-da5b-4585-b153-7a8a05f51b61 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:39.875479 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98a5a3f-da5b-4585-b153-7a8a05f51b61 None admin] PATH_INFO: `/v3/services/network` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:39.876309 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98a5a3f-da5b-4585-b153-7a8a05f51b61 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:39.878019 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98a5a3f-da5b-4585-b153-7a8a05f51b61 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:39.888272 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-e98a5a3f-da5b-4585-b153-7a8a05f51b61 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 09:08:39.888272 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:08:39.888272 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 09:08:39.888272 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 09:08:39.888272 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 09:08:39.888272 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 09:08:39.888272 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 09:08:39.888272 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:08:39.888272 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 09:08:39.888272 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Aug 24 09:08:39.888272 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 09:08:39.888272 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Aug 24 09:08:39.888272 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:08:39.888272 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:08:39.888272 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:08:39.888272 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:08:39.888272 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:08:39.889380 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:08:39.889380 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:08:39.889380 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:08:39.889380 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:08:39.889380 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:08:39.889380 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:08:39.889380 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 09:08:39.889380 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 09:08:39.889380 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 09:08:39.889380 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 09:08:39.889380 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:08:39.889380 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:08:39.889380 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:08:39.889380 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:08:39.889380 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:08:39.889380 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:08:39.890250 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:08:39.890250 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Aug 24 09:08:39.890250 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:08:39.890250 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:08:39.890250 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:08:39.890250 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:08:39.890250 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:08:39.890250 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:08:39.890250 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:08:39.890250 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:08:39.890250 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 09:08:39.890250 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:08:39.890250 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 09:08:39.890250 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Aug 24 09:08:39.893196 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 353/717] 10.222.0.16 () {56 vars in 1111 bytes} [Wed Aug 24 09:08:39 2022] GET /identity/v3/services/network => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:08:39.902925 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f543db52-941c-4728-94ce-9dc380d1109d None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:40.009395 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:40.035925 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f543db52-941c-4728-94ce-9dc380d1109d None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:40.078019 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64883d0c-53a2-4ca7-80b6-daf1ff6cf584 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:40.078811 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64883d0c-53a2-4ca7-80b6-daf1ff6cf584 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:40.079365 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64883d0c-53a2-4ca7-80b6-daf1ff6cf584 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:40.079661 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64883d0c-53a2-4ca7-80b6-daf1ff6cf584 None admin] PATH_INFO: `/v3/services` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:40.080303 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64883d0c-53a2-4ca7-80b6-daf1ff6cf584 None admin] RBAC: Adding query filter params (name=network) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:40.080613 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64883d0c-53a2-4ca7-80b6-daf1ff6cf584 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:40.081988 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64883d0c-53a2-4ca7-80b6-daf1ff6cf584 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:40.091018 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 365/718] 10.222.0.16 () {56 vars in 1112 bytes} [Wed Aug 24 09:08:39 2022] GET /identity/v3/services?name=network => generated 124 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:08:40.096852 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98a5a3f-da5b-4585-b153-7a8a05f51b61 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:40.149927 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:40.236540 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98a5a3f-da5b-4585-b153-7a8a05f51b61 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:40.272153 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d721118-cd81-4ae3-973b-147ef7117762 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:40.272687 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d721118-cd81-4ae3-973b-147ef7117762 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:40.272966 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d721118-cd81-4ae3-973b-147ef7117762 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:40.273238 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d721118-cd81-4ae3-973b-147ef7117762 None admin] PATH_INFO: `/v3/services` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:40.273871 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d721118-cd81-4ae3-973b-147ef7117762 None admin] RBAC: Adding query filter params (type=network) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:40.274176 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d721118-cd81-4ae3-973b-147ef7117762 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:40.275539 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d721118-cd81-4ae3-973b-147ef7117762 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:40.281378 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 354/719] 10.222.0.16 () {56 vars in 1112 bytes} [Wed Aug 24 09:08:40 2022] GET /identity/v3/services?type=network => generated 124 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:40.903313 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96519de9-98b3-4fbd-bb69-0596e7ff0c2f None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:40.903873 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96519de9-98b3-4fbd-bb69-0596e7ff0c2f None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:40.903873 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96519de9-98b3-4fbd-bb69-0596e7ff0c2f None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:40.905522 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 366/720] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:08:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:08:40.911171 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a102950-80ea-40bc-8d54-f7179da74d43 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:40.911645 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a102950-80ea-40bc-8d54-f7179da74d43 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:40.911821 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a102950-80ea-40bc-8d54-f7179da74d43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:40.941354 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-4a102950-80ea-40bc-8d54-f7179da74d43 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:08:40.943506 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-4a102950-80ea-40bc-8d54-f7179da74d43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:41.030020 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 355/721] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:08:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 09:08:41.119609 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecdb42a-b45b-463c-802d-df6927f7ff8b None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:41.119609 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecdb42a-b45b-463c-802d-df6927f7ff8b None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:41.119609 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecdb42a-b45b-463c-802d-df6927f7ff8b None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:41.120999 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 367/722] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:08:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:08:41.125838 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff371443-85b6-4b26-b08d-d5877c22a765 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:41.126124 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff371443-85b6-4b26-b08d-d5877c22a765 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:41.126397 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff371443-85b6-4b26-b08d-d5877c22a765 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:41.148675 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-ff371443-85b6-4b26-b08d-d5877c22a765 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:08:41.149893 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-ff371443-85b6-4b26-b08d-d5877c22a765 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:41.182399 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:41.246321 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 356/723] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:08:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:08:41.355926 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecdb42a-b45b-463c-802d-df6927f7ff8b None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:41.483122 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecdb42a-b45b-463c-802d-df6927f7ff8b None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:41.530140 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbfb442-d48d-40d8-a0f5-773edfda1363 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:41.530771 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbfb442-d48d-40d8-a0f5-773edfda1363 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:41.531046 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbfb442-d48d-40d8-a0f5-773edfda1363 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:41.531312 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbfb442-d48d-40d8-a0f5-773edfda1363 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:41.541681 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:41.548727 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbfb442-d48d-40d8-a0f5-773edfda1363 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:08:41.553398 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-5dbfb442-d48d-40d8-a0f5-773edfda1363 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:08:41.553398 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:08:41.553398 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:08:41.553398 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:08:41.553398 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:08:41.553398 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:08:41.553398 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:08:41.553398 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:08:41.553398 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:08:41.553398 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:08:41.553398 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:08:41.553398 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:08:41.553398 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:08:41.553398 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:08:41.553398 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:08:41.553398 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:08:41.554359 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:08:41.554359 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:08:41.554359 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:08:41.554359 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:08:41.554359 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:08:41.554359 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:08:41.554359 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:08:41.554359 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:08:41.554359 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:08:41.554359 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:08:41.554359 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:08:41.554359 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 24 09:08:41.554359 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:08:41.554359 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:08:41.554359 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:08:41.554359 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:08:41.555246 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:08:41.555246 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:08:41.555246 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:08:41.555246 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:08:41.555246 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:08:41.555246 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:08:41.555246 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:08:41.555246 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:08:41.555246 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:08:41.555246 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:08:41.555246 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:08:41.555246 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 24 09:08:41.555246 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 368/724] 10.222.0.16 () {56 vars in 1096 bytes} [Wed Aug 24 09:08:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:08:41.561078 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff371443-85b6-4b26-b08d-d5877c22a765 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:41.585935 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbfb442-d48d-40d8-a0f5-773edfda1363 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:41.680089 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff371443-85b6-4b26-b08d-d5877c22a765 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:41.733376 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb5d5f4-889b-4e19-8ac4-f775bb853c91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:41.733924 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb5d5f4-889b-4e19-8ac4-f775bb853c91 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:41.734209 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb5d5f4-889b-4e19-8ac4-f775bb853c91 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:41.734488 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb5d5f4-889b-4e19-8ac4-f775bb853c91 None admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:41.735253 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb5d5f4-889b-4e19-8ac4-f775bb853c91 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:41.735560 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb5d5f4-889b-4e19-8ac4-f775bb853c91 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:41.736971 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb5d5f4-889b-4e19-8ac4-f775bb853c91 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:41.748893 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbfb442-d48d-40d8-a0f5-773edfda1363 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:41.753663 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 357/725] 10.222.0.16 () {56 vars in 1099 bytes} [Wed Aug 24 09:08:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:41.761055 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb5d5f4-889b-4e19-8ac4-f775bb853c91 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:41.785590 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ffd2b3-6e7f-4d58-beeb-8ff16cd02b64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:41.786280 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ffd2b3-6e7f-4d58-beeb-8ff16cd02b64 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:41.786560 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ffd2b3-6e7f-4d58-beeb-8ff16cd02b64 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:41.786831 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ffd2b3-6e7f-4d58-beeb-8ff16cd02b64 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:41.797684 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ffd2b3-6e7f-4d58-beeb-8ff16cd02b64 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:08:41.803042 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-79ffd2b3-6e7f-4d58-beeb-8ff16cd02b64 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:08:41.803042 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:08:41.803042 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:08:41.803042 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:08:41.803042 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:08:41.803042 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:08:41.803042 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:08:41.803042 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:08:41.803042 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:08:41.803042 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:08:41.803042 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:08:41.803042 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:08:41.803042 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:08:41.803042 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:08:41.803042 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:08:41.803042 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:08:41.804020 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:08:41.804020 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:08:41.804020 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:08:41.804020 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:08:41.804020 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:08:41.804020 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:08:41.804020 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:08:41.804020 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:08:41.804020 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:08:41.804020 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:08:41.804020 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:08:41.804020 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 24 09:08:41.804020 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:08:41.804020 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:08:41.804020 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:08:41.804020 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:08:41.804887 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:08:41.804887 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:08:41.804887 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:08:41.804887 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:08:41.804887 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:08:41.804887 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:08:41.804887 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:08:41.804887 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:08:41.804887 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:08:41.804887 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:08:41.804887 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:08:41.804887 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 24 09:08:41.805531 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:41.805605 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 369/726] 10.222.0.16 () {56 vars in 1096 bytes} [Wed Aug 24 09:08:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 09:08:41.816992 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ffd2b3-6e7f-4d58-beeb-8ff16cd02b64 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:41.930639 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb5d5f4-889b-4e19-8ac4-f775bb853c91 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:41.939949 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:41.980302 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5ba06a-93bf-4543-8f4f-a34f06e998f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:41.981193 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5ba06a-93bf-4543-8f4f-a34f06e998f7 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:41.981652 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5ba06a-93bf-4543-8f4f-a34f06e998f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:41.982078 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5ba06a-93bf-4543-8f4f-a34f06e998f7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:41.999676 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ffd2b3-6e7f-4d58-beeb-8ff16cd02b64 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:42.002267 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5ba06a-93bf-4543-8f4f-a34f06e998f7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:42.003918 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5ba06a-93bf-4543-8f4f-a34f06e998f7 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:42.030191 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-bd5ba06a-93bf-4543-8f4f-a34f06e998f7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:08:42.030191 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:08:42.030191 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:08:42.030191 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:08:42.030191 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:08:42.030191 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:08:42.030191 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:08:42.030191 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:08:42.030191 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:08:42.030191 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:08:42.030191 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:08:42.030191 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:08:42.030191 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:08:42.030191 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:08:42.030191 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:08:42.030191 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:08:42.031204 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:08:42.031204 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:08:42.031204 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:08:42.031204 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:08:42.031204 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:08:42.031204 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:08:42.031204 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:08:42.031204 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Aug 24 09:08:42.031204 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:08:42.031204 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:08:42.031204 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:08:42.031204 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:08:42.031204 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:08:42.031204 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:08:42.031204 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:08:42.031204 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:08:42.031204 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:08:42.032184 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:08:42.032184 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:08:42.032184 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Aug 24 09:08:42.034331 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 358/727] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 24 09:08:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:08:42.040199 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5ba06a-93bf-4543-8f4f-a34f06e998f7 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:42.042584 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570381cc-6cbf-472e-9d42-07e967e0e012 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:42.043213 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570381cc-6cbf-472e-9d42-07e967e0e012 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:42.043494 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570381cc-6cbf-472e-9d42-07e967e0e012 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:42.043794 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570381cc-6cbf-472e-9d42-07e967e0e012 None admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:42.044462 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570381cc-6cbf-472e-9d42-07e967e0e012 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:42.044785 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570381cc-6cbf-472e-9d42-07e967e0e012 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:42.046845 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570381cc-6cbf-472e-9d42-07e967e0e012 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:42.063630 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 370/728] 10.222.0.16 () {56 vars in 1099 bytes} [Wed Aug 24 09:08:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:08:42.068110 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570381cc-6cbf-472e-9d42-07e967e0e012 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:42.173890 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570381cc-6cbf-472e-9d42-07e967e0e012 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:42.191952 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:42.198533 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:42.212256 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09b24eb-c5df-4c3b-be3f-d06398fd25f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:42.212915 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09b24eb-c5df-4c3b-be3f-d06398fd25f3 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:42.213223 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09b24eb-c5df-4c3b-be3f-d06398fd25f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:42.213495 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09b24eb-c5df-4c3b-be3f-d06398fd25f3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:42.229530 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09b24eb-c5df-4c3b-be3f-d06398fd25f3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:42.230968 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09b24eb-c5df-4c3b-be3f-d06398fd25f3 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:42.236138 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5ba06a-93bf-4543-8f4f-a34f06e998f7 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:42.245774 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-c09b24eb-c5df-4c3b-be3f-d06398fd25f3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:08:42.245774 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:08:42.245774 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:08:42.245774 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:08:42.245774 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:08:42.245774 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:08:42.245774 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:08:42.245774 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:08:42.245774 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:08:42.245774 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:08:42.245774 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:08:42.245774 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:08:42.245774 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:08:42.245774 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:08:42.245774 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:08:42.245774 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:08:42.250129 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:08:42.250129 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:08:42.250129 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:08:42.250129 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:08:42.250129 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:08:42.250129 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:08:42.250129 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:08:42.250129 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 24 09:08:42.250129 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:08:42.250129 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:08:42.250129 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:08:42.250129 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:08:42.250129 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:08:42.250129 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:08:42.250129 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:08:42.250129 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:08:42.250129 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:08:42.251065 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:08:42.251065 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:08:42.251065 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 24 09:08:42.251065 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 371/729] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 24 09:08:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 09:08:42.251065 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a49e3f-9b54-456d-b93f-9f5ac5e05e03 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:42.251065 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a49e3f-9b54-456d-b93f-9f5ac5e05e03 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:42.251065 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a49e3f-9b54-456d-b93f-9f5ac5e05e03 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:42.254536 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 372/730] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:08:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:08:42.255118 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a49e3f-9b54-456d-b93f-9f5ac5e05e03 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:42.281827 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8077ec3-3ad7-4c15-8e58-21746024a85e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:42.283267 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8077ec3-3ad7-4c15-8e58-21746024a85e None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:42.283768 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8077ec3-3ad7-4c15-8e58-21746024a85e None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:42.284193 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8077ec3-3ad7-4c15-8e58-21746024a85e None admin] PATH_INFO: `/v3/domains` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:42.285175 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8077ec3-3ad7-4c15-8e58-21746024a85e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:42.285961 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8077ec3-3ad7-4c15-8e58-21746024a85e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:42.287973 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8077ec3-3ad7-4c15-8e58-21746024a85e None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:42.308597 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 359/731] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Aug 24 09:08:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:42.310331 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469ca4b4-4eba-43bb-ab5e-86e2eb7cc1d3 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:42.310451 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469ca4b4-4eba-43bb-ab5e-86e2eb7cc1d3 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:42.310721 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469ca4b4-4eba-43bb-ab5e-86e2eb7cc1d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:42.334529 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-469ca4b4-4eba-43bb-ab5e-86e2eb7cc1d3 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:08:42.335761 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-469ca4b4-4eba-43bb-ab5e-86e2eb7cc1d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:42.374717 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a49e3f-9b54-456d-b93f-9f5ac5e05e03 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:42.406390 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 360/732] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:08:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:08:42.408700 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469ca4b4-4eba-43bb-ab5e-86e2eb7cc1d3 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:42.411044 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3ebb83-6df4-49fb-ad32-68a2cfba90fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:42.411908 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3ebb83-6df4-49fb-ad32-68a2cfba90fd None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:42.412295 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3ebb83-6df4-49fb-ad32-68a2cfba90fd None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:42.412672 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3ebb83-6df4-49fb-ad32-68a2cfba90fd None admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:42.413825 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3ebb83-6df4-49fb-ad32-68a2cfba90fd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:42.413825 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3ebb83-6df4-49fb-ad32-68a2cfba90fd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:42.415338 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3ebb83-6df4-49fb-ad32-68a2cfba90fd None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:42.429342 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 373/733] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Aug 24 09:08:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:42.434857 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4f4892-4b17-425a-81b7-5ce347d8f473 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:42.435248 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4f4892-4b17-425a-81b7-5ce347d8f473 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:42.435650 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4f4892-4b17-425a-81b7-5ce347d8f473 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:42.437461 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 374/734] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:08:42 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:08:42.442597 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4f4892-4b17-425a-81b7-5ce347d8f473 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:42.483084 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:42.490867 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:42.541960 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469ca4b4-4eba-43bb-ab5e-86e2eb7cc1d3 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:42.565835 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4f4892-4b17-425a-81b7-5ce347d8f473 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:42.579398 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab1d1f2-6f7c-4955-b74d-c84e3b980a02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:42.580250 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab1d1f2-6f7c-4955-b74d-c84e3b980a02 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:42.580630 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab1d1f2-6f7c-4955-b74d-c84e3b980a02 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:42.581025 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab1d1f2-6f7c-4955-b74d-c84e3b980a02 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:42.589574 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab1d1f2-6f7c-4955-b74d-c84e3b980a02 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:42.591155 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab1d1f2-6f7c-4955-b74d-c84e3b980a02 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:42.603542 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-8ab1d1f2-6f7c-4955-b74d-c84e3b980a02 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 09:08:42.603542 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:08:42.603542 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:08:42.603542 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:08:42.603542 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:08:42.603542 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:08:42.603542 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:08:42.603542 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:08:42.603542 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:08:42.603542 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:08:42.603542 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:08:42.603542 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:08:42.603542 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 09:08:42.603542 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 09:08:42.603542 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 09:08:42.603542 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 09:08:42.605353 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:08:42.605353 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:08:42.605353 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 09:08:42.605353 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:08:42.605353 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 09:08:42.605353 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:08:42.605353 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:08:42.605353 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:08:42.605353 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:08:42.605353 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:08:42.605353 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:08:42.605353 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Aug 24 09:08:42.605353 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:08:42.605353 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:08:42.605353 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:08:42.605353 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:08:42.605353 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:08:42.605353 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:08:42.606541 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:08:42.606541 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:08:42.606541 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 09:08:42.606541 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 09:08:42.606541 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 09:08:42.606541 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 09:08:42.606541 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 09:08:42.606541 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 09:08:42.606541 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 09:08:42.606541 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Aug 24 09:08:42.611999 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 361/735] 10.222.0.16 () {56 vars in 1096 bytes} [Wed Aug 24 09:08:42 2022] GET /identity/v3/users/swift => generated 83 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:08:42.612754 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a9936d-d750-47a5-aaef-a05e8da8e55c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:42.613529 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403c4073-8385-41f6-9405-fd87cb28afda None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:42.613700 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a9936d-d750-47a5-aaef-a05e8da8e55c None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:42.614002 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403c4073-8385-41f6-9405-fd87cb28afda None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:42.614377 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403c4073-8385-41f6-9405-fd87cb28afda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:42.614489 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a9936d-d750-47a5-aaef-a05e8da8e55c None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:42.614885 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a9936d-d750-47a5-aaef-a05e8da8e55c None admin] PATH_INFO: `/v3/users/nova` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:42.624233 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a9936d-d750-47a5-aaef-a05e8da8e55c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:42.625882 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a9936d-d750-47a5-aaef-a05e8da8e55c None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:42.634700 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-403c4073-8385-41f6-9405-fd87cb28afda None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:08:42.636776 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-403c4073-8385-41f6-9405-fd87cb28afda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:42.640743 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-95a9936d-d750-47a5-aaef-a05e8da8e55c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 09:08:42.640743 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:08:42.640743 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:08:42.640743 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:08:42.640743 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:08:42.640743 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:08:42.640743 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:08:42.640743 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:08:42.640743 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:08:42.640743 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:08:42.640743 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:08:42.640743 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:08:42.640743 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 09:08:42.640743 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 09:08:42.640743 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 09:08:42.640743 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 09:08:42.641881 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:08:42.641881 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:08:42.641881 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 09:08:42.641881 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:08:42.641881 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 09:08:42.641881 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:08:42.641881 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:08:42.641881 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:08:42.641881 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:08:42.641881 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:08:42.641881 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:08:42.641881 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 24 09:08:42.641881 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:08:42.641881 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:08:42.641881 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:08:42.641881 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:08:42.641881 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:08:42.641881 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:08:42.642866 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:08:42.642866 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:08:42.642866 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 09:08:42.642866 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 09:08:42.642866 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 09:08:42.642866 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 09:08:42.642866 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 09:08:42.642866 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 09:08:42.642866 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 09:08:42.642866 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 24 09:08:42.645191 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 375/736] 10.222.0.16 () {56 vars in 1093 bytes} [Wed Aug 24 09:08:42 2022] GET /identity/v3/users/nova => generated 82 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:08:42.646154 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a9936d-d750-47a5-aaef-a05e8da8e55c None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:42.716168 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 362/737] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:08:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:08:42.716989 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403c4073-8385-41f6-9405-fd87cb28afda None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:42.768110 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:42.772374 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:42.775610 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a9936d-d750-47a5-aaef-a05e8da8e55c None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:42.811228 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df22db14-aff0-4cd9-b922-79cb9aba654c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:42.812065 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df22db14-aff0-4cd9-b922-79cb9aba654c None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:42.812348 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df22db14-aff0-4cd9-b922-79cb9aba654c None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:42.812619 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df22db14-aff0-4cd9-b922-79cb9aba654c None admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:42.813510 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df22db14-aff0-4cd9-b922-79cb9aba654c None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:42.814132 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df22db14-aff0-4cd9-b922-79cb9aba654c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:42.816025 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df22db14-aff0-4cd9-b922-79cb9aba654c None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:42.822026 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403c4073-8385-41f6-9405-fd87cb28afda None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:42.842776 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 376/738] 10.222.0.16 () {56 vars in 1135 bytes} [Wed Aug 24 09:08:42 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:42.842776 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df22db14-aff0-4cd9-b922-79cb9aba654c None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:42.861257 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25657b07-905d-4d75-830d-57c52ee50145 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:42.861867 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25657b07-905d-4d75-830d-57c52ee50145 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:42.862148 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25657b07-905d-4d75-830d-57c52ee50145 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:42.862423 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25657b07-905d-4d75-830d-57c52ee50145 None admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:42.863156 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25657b07-905d-4d75-830d-57c52ee50145 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:42.863462 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25657b07-905d-4d75-830d-57c52ee50145 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:42.864925 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25657b07-905d-4d75-830d-57c52ee50145 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:42.897613 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 363/739] 10.222.0.16 () {56 vars in 1133 bytes} [Wed Aug 24 09:08:42 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:42.898949 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b794c1ae-59d4-475e-9d85-0d0747178804 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:42.899234 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b794c1ae-59d4-475e-9d85-0d0747178804 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:42.899507 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b794c1ae-59d4-475e-9d85-0d0747178804 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:42.903714 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 364/740] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:08:42 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:08:42.905396 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b794c1ae-59d4-475e-9d85-0d0747178804 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:42.950903 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df22db14-aff0-4cd9-b922-79cb9aba654c None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:42.986689 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5595bad3-8199-49c9-a00d-560c4066a31d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:42.987572 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5595bad3-8199-49c9-a00d-560c4066a31d None admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:42.988029 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5595bad3-8199-49c9-a00d-560c4066a31d None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:42.988468 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5595bad3-8199-49c9-a00d-560c4066a31d None admin] PATH_INFO: `/v3/services` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:42.989535 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5595bad3-8199-49c9-a00d-560c4066a31d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:42.991059 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5595bad3-8199-49c9-a00d-560c4066a31d None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:43.013973 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b794c1ae-59d4-475e-9d85-0d0747178804 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:43.023243 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:43.048162 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d562f4-11b0-424a-b6fa-38163f0e4b44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:43.048766 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d562f4-11b0-424a-b6fa-38163f0e4b44 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:43.049047 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d562f4-11b0-424a-b6fa-38163f0e4b44 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:43.049323 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d562f4-11b0-424a-b6fa-38163f0e4b44 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:43.098390 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 377/741] 10.222.0.16 () {60 vars in 1141 bytes} [Wed Aug 24 09:08:42 2022] POST /identity/v3/services => generated 251 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:08:43.099159 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5595bad3-8199-49c9-a00d-560c4066a31d None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:43.115821 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:43.120824 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d562f4-11b0-424a-b6fa-38163f0e4b44 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:43.122320 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d562f4-11b0-424a-b6fa-38163f0e4b44 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:43.224694 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5595bad3-8199-49c9-a00d-560c4066a31d None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:43.238358 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 365/742] 10.222.0.16 () {58 vars in 1282 bytes} [Wed Aug 24 09:08:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 09:08:43.239650 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6a7435-612b-408c-91b0-e59f88833302 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:43.239939 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6a7435-612b-408c-91b0-e59f88833302 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:43.240266 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6a7435-612b-408c-91b0-e59f88833302 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:43.258366 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-1e6a7435-612b-408c-91b0-e59f88833302 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:08:43.260165 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-1e6a7435-612b-408c-91b0-e59f88833302 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:43.262098 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:43.267435 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9e369b-5c60-4096-a072-adee92248fd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:43.270782 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9e369b-5c60-4096-a072-adee92248fd7 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:43.270782 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9e369b-5c60-4096-a072-adee92248fd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:43.270782 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9e369b-5c60-4096-a072-adee92248fd7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:43.279606 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9e369b-5c60-4096-a072-adee92248fd7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:43.281172 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9e369b-5c60-4096-a072-adee92248fd7 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:43.298458 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-5d9e369b-5c60-4096-a072-adee92248fd7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:08:43.298458 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:08:43.298458 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:08:43.298458 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:08:43.298458 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:08:43.298458 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:08:43.298458 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:08:43.298458 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:08:43.298458 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:08:43.298458 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:08:43.298458 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:08:43.298458 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:08:43.298458 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:08:43.298458 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:08:43.298458 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:08:43.298458 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:08:43.299467 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:08:43.299467 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:08:43.299467 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:08:43.299467 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:08:43.299467 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:08:43.299467 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:08:43.299467 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:08:43.299467 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 24 09:08:43.299467 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:08:43.299467 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:08:43.299467 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:08:43.299467 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:08:43.299467 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:08:43.299467 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:08:43.299467 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:08:43.299467 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:08:43.299467 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:08:43.300411 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:08:43.300411 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:08:43.300411 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 24 09:08:43.300411 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 378/743] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 24 09:08:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:08:43.305553 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9e369b-5c60-4096-a072-adee92248fd7 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:43.337807 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 366/744] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:08:43 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:08:43.339195 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6a7435-612b-408c-91b0-e59f88833302 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:43.388023 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:43.445604 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6a7435-612b-408c-91b0-e59f88833302 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:43.448796 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9e369b-5c60-4096-a072-adee92248fd7 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:43.482633 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50be0e29-7ea9-41a8-ba6d-2f578ea3b55f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:43.483365 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50be0e29-7ea9-41a8-ba6d-2f578ea3b55f None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:43.483785 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50be0e29-7ea9-41a8-ba6d-2f578ea3b55f None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:43.484126 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50be0e29-7ea9-41a8-ba6d-2f578ea3b55f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:43.491348 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:43.495848 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50be0e29-7ea9-41a8-ba6d-2f578ea3b55f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:43.496914 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4fc8e5-583a-47d0-83df-4c8db3e21ce5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:43.497482 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50be0e29-7ea9-41a8-ba6d-2f578ea3b55f None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:43.498044 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4fc8e5-583a-47d0-83df-4c8db3e21ce5 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:43.498560 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4fc8e5-583a-47d0-83df-4c8db3e21ce5 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:43.499010 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4fc8e5-583a-47d0-83df-4c8db3e21ce5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:43.511618 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4fc8e5-583a-47d0-83df-4c8db3e21ce5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:08:43.513179 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-50be0e29-7ea9-41a8-ba6d-2f578ea3b55f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:08:43.513179 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:08:43.513179 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:08:43.513179 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:08:43.513179 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:08:43.513179 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:08:43.513179 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:08:43.513179 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:08:43.513179 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:08:43.513179 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:08:43.513179 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:08:43.513179 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:08:43.513179 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:08:43.513179 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:08:43.513179 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:08:43.513179 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:08:43.514305 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:08:43.514305 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:08:43.514305 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:08:43.514305 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:08:43.514305 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:08:43.514305 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:08:43.514305 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:08:43.514305 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Aug 24 09:08:43.514305 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:08:43.514305 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:08:43.514305 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:08:43.514305 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:08:43.514305 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:08:43.514305 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:08:43.514305 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:08:43.514305 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:08:43.514305 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:08:43.515402 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:08:43.515402 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:08:43.515402 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Aug 24 09:08:43.515822 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 367/745] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 24 09:08:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:08:43.516818 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50be0e29-7ea9-41a8-ba6d-2f578ea3b55f None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:43.521710 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-ef4fc8e5-583a-47d0-83df-4c8db3e21ce5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 09:08:43.521710 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:08:43.521710 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:08:43.521710 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:08:43.521710 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:08:43.521710 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:08:43.521710 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:08:43.521710 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:08:43.521710 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:08:43.521710 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:08:43.521710 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:08:43.521710 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:08:43.521710 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:08:43.521710 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:08:43.521710 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:08:43.521710 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:08:43.522706 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:08:43.522706 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:08:43.522706 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:08:43.522706 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:08:43.522706 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:08:43.522706 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:08:43.522706 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:08:43.522706 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:08:43.522706 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:08:43.522706 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:08:43.522706 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:08:43.522706 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 24 09:08:43.522706 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:08:43.522706 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:08:43.522706 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:08:43.522706 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:08:43.523657 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:08:43.523657 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:08:43.523657 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:08:43.523657 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:08:43.523657 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:08:43.523657 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:08:43.523657 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:08:43.523657 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:08:43.523657 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:08:43.523657 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:08:43.523657 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 09:08:43.523657 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 24 09:08:43.526303 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 379/746] 10.222.0.16 () {56 vars in 1102 bytes} [Wed Aug 24 09:08:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:08:43.531478 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4fc8e5-583a-47d0-83df-4c8db3e21ce5 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:43.647786 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4fc8e5-583a-47d0-83df-4c8db3e21ce5 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:43.650501 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50be0e29-7ea9-41a8-ba6d-2f578ea3b55f None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:43.668866 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:43.683780 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e03fb6-736e-4ff5-a4aa-88f11398b590 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:43.684391 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e03fb6-736e-4ff5-a4aa-88f11398b590 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:43.684669 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e03fb6-736e-4ff5-a4aa-88f11398b590 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:43.684941 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e03fb6-736e-4ff5-a4aa-88f11398b590 None admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:43.685581 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e03fb6-736e-4ff5-a4aa-88f11398b590 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:43.685890 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e03fb6-736e-4ff5-a4aa-88f11398b590 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:43.686706 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2e8807-3bc4-44f3-bda1-896a58b015f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:43.687268 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e03fb6-736e-4ff5-a4aa-88f11398b590 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:43.687380 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2e8807-3bc4-44f3-bda1-896a58b015f6 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:43.688589 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2e8807-3bc4-44f3-bda1-896a58b015f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:43.688902 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2e8807-3bc4-44f3-bda1-896a58b015f6 None admin] PATH_INFO: `/v3/domains` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:43.689686 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2e8807-3bc4-44f3-bda1-896a58b015f6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:43.689871 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2e8807-3bc4-44f3-bda1-896a58b015f6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:43.691271 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2e8807-3bc4-44f3-bda1-896a58b015f6 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:43.704816 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 380/747] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Aug 24 09:08:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:43.704816 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e03fb6-736e-4ff5-a4aa-88f11398b590 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:43.705029 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 368/748] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Aug 24 09:08:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:43.705217 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2e8807-3bc4-44f3-bda1-896a58b015f6 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:43.759759 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:43.810198 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e03fb6-736e-4ff5-a4aa-88f11398b590 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:43.811442 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2e8807-3bc4-44f3-bda1-896a58b015f6 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:43.848260 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7bc92a-e8de-49ef-80a1-5639dfb6bd7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:43.848873 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7bc92a-e8de-49ef-80a1-5639dfb6bd7c None admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:43.849152 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7bc92a-e8de-49ef-80a1-5639dfb6bd7c None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:43.849266 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b4e294-f4bb-41fd-8c81-bc19de2bef6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:43.849444 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7bc92a-e8de-49ef-80a1-5639dfb6bd7c None admin] PATH_INFO: `/v3/services` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:43.849797 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b4e294-f4bb-41fd-8c81-bc19de2bef6e None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:43.850063 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7bc92a-e8de-49ef-80a1-5639dfb6bd7c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:43.850172 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b4e294-f4bb-41fd-8c81-bc19de2bef6e None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:43.850333 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b4e294-f4bb-41fd-8c81-bc19de2bef6e None admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:43.850991 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b4e294-f4bb-41fd-8c81-bc19de2bef6e None admin] RBAC: Adding query filter params (name=service) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:43.851296 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b4e294-f4bb-41fd-8c81-bc19de2bef6e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:43.851452 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7bc92a-e8de-49ef-80a1-5639dfb6bd7c None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:43.852691 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b4e294-f4bb-41fd-8c81-bc19de2bef6e None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:43.861570 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 381/749] 10.222.0.16 () {56 vars in 1103 bytes} [Wed Aug 24 09:08:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:43.862357 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b4e294-f4bb-41fd-8c81-bc19de2bef6e None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:43.905293 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:43.927784 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 369/750] 10.222.0.16 () {60 vars in 1141 bytes} [Wed Aug 24 09:08:43 2022] POST /identity/v3/services => generated 240 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:08:43.927784 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7bc92a-e8de-49ef-80a1-5639dfb6bd7c None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:43.976061 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b4e294-f4bb-41fd-8c81-bc19de2bef6e None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:44.034639 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b88347-9e69-49d8-a798-0ed5ab730af9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:44.035287 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b88347-9e69-49d8-a798-0ed5ab730af9 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:44.035571 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b88347-9e69-49d8-a798-0ed5ab730af9 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:44.035882 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b88347-9e69-49d8-a798-0ed5ab730af9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:44.047013 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b88347-9e69-49d8-a798-0ed5ab730af9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:44.048872 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b88347-9e69-49d8-a798-0ed5ab730af9 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:44.053988 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7bc92a-e8de-49ef-80a1-5639dfb6bd7c None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:44.064413 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-d1b88347-9e69-49d8-a798-0ed5ab730af9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:08:44.064413 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:08:44.064413 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:08:44.064413 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:08:44.064413 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:08:44.064413 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:08:44.064413 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:08:44.064413 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:08:44.064413 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:08:44.064413 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:08:44.064413 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:08:44.064413 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:08:44.064413 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 09:08:44.064413 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 09:08:44.064413 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 09:08:44.065430 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 09:08:44.065430 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:08:44.065430 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:08:44.065430 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:08:44.065430 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:08:44.065430 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:08:44.065430 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:08:44.065430 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:08:44.065430 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 24 09:08:44.065430 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:08:44.065430 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:08:44.065430 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:08:44.065430 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:08:44.065430 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:08:44.065430 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:08:44.065430 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:08:44.065430 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:08:44.066325 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 09:08:44.066325 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 09:08:44.066325 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 09:08:44.066325 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 09:08:44.066325 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 09:08:44.066325 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 09:08:44.066325 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:08:44.066325 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 24 09:08:44.072567 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 382/751] 10.222.0.16 () {56 vars in 1111 bytes} [Wed Aug 24 09:08:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:08:44.072567 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b88347-9e69-49d8-a798-0ed5ab730af9 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:44.072786 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:44.110253 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33edd5cf-8e0f-4882-b75a-3833d3b9937b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:44.110903 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33edd5cf-8e0f-4882-b75a-3833d3b9937b None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:44.111187 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33edd5cf-8e0f-4882-b75a-3833d3b9937b None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:44.111463 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33edd5cf-8e0f-4882-b75a-3833d3b9937b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:44.122029 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33edd5cf-8e0f-4882-b75a-3833d3b9937b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:44.123489 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33edd5cf-8e0f-4882-b75a-3833d3b9937b None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:44.137052 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-33edd5cf-8e0f-4882-b75a-3833d3b9937b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:08:44.137052 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:08:44.137052 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:08:44.137052 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:08:44.137052 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:08:44.137052 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:08:44.137052 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:08:44.137052 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:08:44.137052 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:08:44.137052 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:08:44.137052 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:08:44.137052 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:08:44.137052 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 09:08:44.137052 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 09:08:44.137052 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 09:08:44.138009 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 09:08:44.138009 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:08:44.138009 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:08:44.138009 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:08:44.138009 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:08:44.138009 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:08:44.138009 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:08:44.138009 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:08:44.138009 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Aug 24 09:08:44.138009 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:08:44.138009 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:08:44.138009 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:08:44.138009 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:08:44.138009 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:08:44.138009 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:08:44.138009 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:08:44.138009 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:08:44.139842 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 09:08:44.139842 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 09:08:44.139842 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 09:08:44.139842 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 09:08:44.139842 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 09:08:44.139842 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 09:08:44.139842 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:08:44.139842 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Aug 24 09:08:44.139842 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 370/752] 10.222.0.16 () {56 vars in 1111 bytes} [Wed Aug 24 09:08:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:08:44.139842 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33edd5cf-8e0f-4882-b75a-3833d3b9937b None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:44.295875 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:44.307062 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b88347-9e69-49d8-a798-0ed5ab730af9 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:44.340444 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33edd5cf-8e0f-4882-b75a-3833d3b9937b None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:44.352530 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f3b454-c358-4ba9-a5ad-53860688dc5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:44.353511 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f3b454-c358-4ba9-a5ad-53860688dc5e None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:44.354121 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f3b454-c358-4ba9-a5ad-53860688dc5e None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:44.354621 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f3b454-c358-4ba9-a5ad-53860688dc5e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:44.365997 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f3b454-c358-4ba9-a5ad-53860688dc5e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:44.367918 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f3b454-c358-4ba9-a5ad-53860688dc5e None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:44.375985 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990acf5b-622f-4790-8c17-2b8e0a3c94a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:44.376647 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990acf5b-622f-4790-8c17-2b8e0a3c94a7 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:44.376929 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990acf5b-622f-4790-8c17-2b8e0a3c94a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:44.377202 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990acf5b-622f-4790-8c17-2b8e0a3c94a7 None admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:44.377943 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990acf5b-622f-4790-8c17-2b8e0a3c94a7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:44.378249 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990acf5b-622f-4790-8c17-2b8e0a3c94a7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:44.379719 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990acf5b-622f-4790-8c17-2b8e0a3c94a7 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:44.386228 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-10f3b454-c358-4ba9-a5ad-53860688dc5e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:08:44.386228 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:08:44.386228 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:08:44.386228 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:08:44.386228 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:08:44.386228 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:08:44.386228 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:08:44.386228 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:08:44.386228 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:08:44.386228 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:08:44.386228 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:08:44.386228 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:08:44.386228 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:08:44.386228 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:08:44.386228 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:08:44.386228 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:08:44.387180 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:08:44.387180 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:08:44.387180 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:08:44.387180 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:08:44.387180 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:08:44.387180 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:08:44.387180 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:08:44.387180 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 24 09:08:44.387180 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:08:44.387180 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:08:44.387180 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:08:44.387180 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:08:44.387180 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:08:44.387180 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:08:44.387180 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:08:44.387180 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:08:44.387180 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:08:44.388127 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:08:44.388127 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:08:44.388127 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 24 09:08:44.390328 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 383/753] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 24 09:08:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 319 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:08:44.391461 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f3b454-c358-4ba9-a5ad-53860688dc5e None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:44.392897 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 371/754] 10.222.0.16 () {56 vars in 1148 bytes} [Wed Aug 24 09:08:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:44.395435 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990acf5b-622f-4790-8c17-2b8e0a3c94a7 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:44.428743 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:44.503965 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f3b454-c358-4ba9-a5ad-53860688dc5e None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:44.512045 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990acf5b-622f-4790-8c17-2b8e0a3c94a7 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:44.539044 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c692da-b52b-4b5f-9e2a-5a3170b50ca0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:44.539822 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c692da-b52b-4b5f-9e2a-5a3170b50ca0 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:44.540202 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c692da-b52b-4b5f-9e2a-5a3170b50ca0 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:44.540581 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c692da-b52b-4b5f-9e2a-5a3170b50ca0 None admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:44.541973 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c692da-b52b-4b5f-9e2a-5a3170b50ca0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:44.542380 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c692da-b52b-4b5f-9e2a-5a3170b50ca0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:44.543916 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c692da-b52b-4b5f-9e2a-5a3170b50ca0 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:44.551084 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ff7f29-1b7c-4c8a-8387-7d7718b1c938 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:44.552027 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ff7f29-1b7c-4c8a-8387-7d7718b1c938 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:44.552474 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ff7f29-1b7c-4c8a-8387-7d7718b1c938 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:44.553019 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ff7f29-1b7c-4c8a-8387-7d7718b1c938 None admin] PATH_INFO: `/v3/domains` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:44.553919 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ff7f29-1b7c-4c8a-8387-7d7718b1c938 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:44.554360 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ff7f29-1b7c-4c8a-8387-7d7718b1c938 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:44.555893 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ff7f29-1b7c-4c8a-8387-7d7718b1c938 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:44.557481 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 384/755] 10.222.0.16 () {56 vars in 1148 bytes} [Wed Aug 24 09:08:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:44.558435 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c692da-b52b-4b5f-9e2a-5a3170b50ca0 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:44.569999 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 372/756] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Aug 24 09:08:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:44.571242 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:44.572249 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ff7f29-1b7c-4c8a-8387-7d7718b1c938 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:44.673919 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c692da-b52b-4b5f-9e2a-5a3170b50ca0 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:44.700053 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:44.712557 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ff7f29-1b7c-4c8a-8387-7d7718b1c938 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:44.715344 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f690c3e3-2d83-44c3-8411-c0724bb42d9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:44.716777 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f690c3e3-2d83-44c3-8411-c0724bb42d9e None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:44.717065 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f690c3e3-2d83-44c3-8411-c0724bb42d9e None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:44.717341 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f690c3e3-2d83-44c3-8411-c0724bb42d9e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:44.718071 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f690c3e3-2d83-44c3-8411-c0724bb42d9e None admin] RBAC: Adding query filter params (role.id=15f634d4a2d247ddab90c458e5e1c141, scope.project.id=c467e88e2fb0491bacefcf6d2ce326d9, user.id=fdd5f0d67fe54e9fae6f4af644988d93) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:44.718380 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f690c3e3-2d83-44c3-8411-c0724bb42d9e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:44.719824 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f690c3e3-2d83-44c3-8411-c0724bb42d9e None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:44.727439 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 385/757] 10.222.0.16 () {56 vars in 1374 bytes} [Wed Aug 24 09:08:44 2022] GET /identity/v3/role_assignments?user.id=fdd5f0d67fe54e9fae6f4af644988d93&scope.project.id=c467e88e2fb0491bacefcf6d2ce326d9&role.id=15f634d4a2d247ddab90c458e5e1c141 => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:44.728487 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f690c3e3-2d83-44c3-8411-c0724bb42d9e None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:44.751230 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46a2269-8c97-4802-844b-86881d4ccc03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:44.752333 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46a2269-8c97-4802-844b-86881d4ccc03 None admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:44.753186 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46a2269-8c97-4802-844b-86881d4ccc03 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:44.754178 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46a2269-8c97-4802-844b-86881d4ccc03 None admin] PATH_INFO: `/v3/projects/c467e88e2fb0491bacefcf6d2ce326d9/users/1f055f7dc51d43bda9b315ca67f11d58/roles/15f634d4a2d247ddab90c458e5e1c141` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:44.756920 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46a2269-8c97-4802-844b-86881d4ccc03 None admin] RBAC: Authorizing `identity:create_grant(project_id=c467e88e2fb0491bacefcf6d2ce326d9, user_id=1f055f7dc51d43bda9b315ca67f11d58, role_id=15f634d4a2d247ddab90c458e5e1c141)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:44.758694 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46a2269-8c97-4802-844b-86881d4ccc03 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:44.825710 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:44.854594 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f690c3e3-2d83-44c3-8411-c0724bb42d9e None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:44.894428 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b665472-6bba-4341-8281-5556fc71ca94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:44.896746 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b665472-6bba-4341-8281-5556fc71ca94 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:44.896746 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b665472-6bba-4341-8281-5556fc71ca94 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:44.896746 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b665472-6bba-4341-8281-5556fc71ca94 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:44.897281 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 373/758] 10.222.0.16 () {58 vars in 1439 bytes} [Wed Aug 24 09:08:44 2022] PUT /identity/v3/projects/c467e88e2fb0491bacefcf6d2ce326d9/users/1f055f7dc51d43bda9b315ca67f11d58/roles/15f634d4a2d247ddab90c458e5e1c141 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:08:44.905481 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b665472-6bba-4341-8281-5556fc71ca94 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:44.907017 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b665472-6bba-4341-8281-5556fc71ca94 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:44.921262 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-9b665472-6bba-4341-8281-5556fc71ca94 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 09:08:44.921262 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:08:44.921262 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:08:44.921262 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:08:44.921262 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:08:44.921262 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:08:44.921262 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:08:44.921262 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:08:44.921262 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:08:44.921262 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:08:44.921262 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:08:44.921262 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:08:44.921262 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 09:08:44.921262 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 09:08:44.921262 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 09:08:44.921262 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 09:08:44.922508 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:08:44.922508 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:08:44.922508 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 09:08:44.922508 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:08:44.922508 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 09:08:44.922508 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:08:44.922508 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:08:44.922508 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:08:44.922508 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:08:44.922508 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:08:44.922508 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:08:44.922508 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 24 09:08:44.922508 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:08:44.922508 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:08:44.922508 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:08:44.922508 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:08:44.922508 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:08:44.922508 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:08:44.923730 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:08:44.923730 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:08:44.923730 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 09:08:44.923730 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 09:08:44.923730 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 09:08:44.923730 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 09:08:44.923730 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 09:08:44.923730 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 09:08:44.923730 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 09:08:44.923730 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 24 09:08:44.929592 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 386/759] 10.222.0.16 () {56 vars in 1117 bytes} [Wed Aug 24 09:08:44 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:08:44.936509 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46a2269-8c97-4802-844b-86881d4ccc03 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:45.051502 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46a2269-8c97-4802-844b-86881d4ccc03 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:45.088834 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c044fcec-05db-44f0-8b1a-39db84c48768 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:45.089437 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c044fcec-05db-44f0-8b1a-39db84c48768 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:45.089692 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c044fcec-05db-44f0-8b1a-39db84c48768 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:45.089968 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c044fcec-05db-44f0-8b1a-39db84c48768 None admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:45.090769 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c044fcec-05db-44f0-8b1a-39db84c48768 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:45.091084 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c044fcec-05db-44f0-8b1a-39db84c48768 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:45.092547 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c044fcec-05db-44f0-8b1a-39db84c48768 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:45.114474 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 374/760] 10.222.0.16 () {56 vars in 1149 bytes} [Wed Aug 24 09:08:44 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:45.120606 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b665472-6bba-4341-8281-5556fc71ca94 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:45.168203 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12400d7e-9d7d-4e60-8947-d97f85a342d3 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:45.168682 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12400d7e-9d7d-4e60-8947-d97f85a342d3 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:45.169058 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12400d7e-9d7d-4e60-8947-d97f85a342d3 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:45.170788 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 375/761] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:08:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 09:08:45.179948 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be65c17-5998-4bb1-bbc2-43309a6afd31 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:45.180360 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be65c17-5998-4bb1-bbc2-43309a6afd31 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:45.180739 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be65c17-5998-4bb1-bbc2-43309a6afd31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:45.220499 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-2be65c17-5998-4bb1-bbc2-43309a6afd31 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:08:45.221931 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-2be65c17-5998-4bb1-bbc2-43309a6afd31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:45.224249 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:45.232841 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b665472-6bba-4341-8281-5556fc71ca94 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:45.268650 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f17d488-baa4-4c5d-a44d-5e8a67218418 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:45.269534 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f17d488-baa4-4c5d-a44d-5e8a67218418 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:45.269959 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f17d488-baa4-4c5d-a44d-5e8a67218418 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:45.270372 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f17d488-baa4-4c5d-a44d-5e8a67218418 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:45.296097 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:45.345186 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f17d488-baa4-4c5d-a44d-5e8a67218418 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:45.346955 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f17d488-baa4-4c5d-a44d-5e8a67218418 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:45.359423 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 376/762] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:08:45 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 09:08:45.448911 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 387/763] 10.222.0.16 () {58 vars in 1282 bytes} [Wed Aug 24 09:08:45 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 09:08:45.454820 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be65c17-5998-4bb1-bbc2-43309a6afd31 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:45.570594 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be65c17-5998-4bb1-bbc2-43309a6afd31 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:45.621273 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fffe9c-33c1-4307-929f-ac1c6a63d635 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:45.621841 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fffe9c-33c1-4307-929f-ac1c6a63d635 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:45.622125 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fffe9c-33c1-4307-929f-ac1c6a63d635 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:45.622397 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fffe9c-33c1-4307-929f-ac1c6a63d635 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:45.629310 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:45.633758 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fffe9c-33c1-4307-929f-ac1c6a63d635 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:45.635770 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fffe9c-33c1-4307-929f-ac1c6a63d635 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:45.651140 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-a6fffe9c-33c1-4307-929f-ac1c6a63d635 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:08:45.651140 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:08:45.651140 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:08:45.651140 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:08:45.651140 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:08:45.651140 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:08:45.651140 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:08:45.651140 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:08:45.651140 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:08:45.651140 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:08:45.651140 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:08:45.651140 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:08:45.651140 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:08:45.651140 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:08:45.651140 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:08:45.651140 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:08:45.652161 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:08:45.652161 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:08:45.652161 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:08:45.652161 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:08:45.652161 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:08:45.652161 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:08:45.652161 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:08:45.652161 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Aug 24 09:08:45.652161 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:08:45.652161 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:08:45.652161 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:08:45.652161 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:08:45.652161 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:08:45.652161 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:08:45.652161 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:08:45.652161 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:08:45.652161 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:08:45.653090 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:08:45.653090 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:08:45.653090 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Aug 24 09:08:45.653090 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 377/764] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 24 09:08:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:08:45.658795 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f17d488-baa4-4c5d-a44d-5e8a67218418 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:45.703296 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:45.771599 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f17d488-baa4-4c5d-a44d-5e8a67218418 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:45.773729 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fffe9c-33c1-4307-929f-ac1c6a63d635 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:45.806999 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d16208-aee5-41dd-9f8b-c41940c8c704 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:45.808050 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d16208-aee5-41dd-9f8b-c41940c8c704 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:45.808576 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d16208-aee5-41dd-9f8b-c41940c8c704 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:45.809068 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d16208-aee5-41dd-9f8b-c41940c8c704 None admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:45.809916 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d16208-aee5-41dd-9f8b-c41940c8c704 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:45.810427 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d16208-aee5-41dd-9f8b-c41940c8c704 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:45.812007 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d16208-aee5-41dd-9f8b-c41940c8c704 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:45.825242 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 388/765] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Aug 24 09:08:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:45.831131 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d16208-aee5-41dd-9f8b-c41940c8c704 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:45.932860 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d16208-aee5-41dd-9f8b-c41940c8c704 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:45.935057 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:45.941359 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fffe9c-33c1-4307-929f-ac1c6a63d635 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:45.996482 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f14c67-eed4-43ed-94e6-01461b3dc2ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:45.998135 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f14c67-eed4-43ed-94e6-01461b3dc2ca None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:45.998779 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f14c67-eed4-43ed-94e6-01461b3dc2ca None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:46.000294 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f14c67-eed4-43ed-94e6-01461b3dc2ca None admin] PATH_INFO: `/v3/projects/service` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:46.010613 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f14c67-eed4-43ed-94e6-01461b3dc2ca None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:46.012813 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b5cb94-a143-4627-b2f5-7016761ba3e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:46.013688 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b5cb94-a143-4627-b2f5-7016761ba3e5 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:46.014174 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b5cb94-a143-4627-b2f5-7016761ba3e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:46.014664 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b5cb94-a143-4627-b2f5-7016761ba3e5 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:46.014914 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f14c67-eed4-43ed-94e6-01461b3dc2ca None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:46.015850 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b5cb94-a143-4627-b2f5-7016761ba3e5 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:46.017404 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b5cb94-a143-4627-b2f5-7016761ba3e5 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:46.027245 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-a7b5cb94-a143-4627-b2f5-7016761ba3e5 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 09:08:46.027245 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:08:46.027245 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 09:08:46.027245 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 09:08:46.027245 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 09:08:46.027245 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 09:08:46.027245 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 09:08:46.027245 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:08:46.027245 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 09:08:46.027245 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Aug 24 09:08:46.027245 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 09:08:46.027245 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Aug 24 09:08:46.027245 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:08:46.027245 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:08:46.027245 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:08:46.027245 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:08:46.027245 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:08:46.028266 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:08:46.028266 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:08:46.028266 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:08:46.028266 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:08:46.028266 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:08:46.028266 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:08:46.028266 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 09:08:46.028266 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 09:08:46.028266 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 09:08:46.028266 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 09:08:46.028266 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:08:46.028266 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:08:46.028266 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:08:46.028266 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:08:46.028266 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:08:46.028266 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:08:46.029205 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:08:46.029205 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Aug 24 09:08:46.029205 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:08:46.029205 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:08:46.029205 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:08:46.029205 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:08:46.029205 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:08:46.029205 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:08:46.029205 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:08:46.029205 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:08:46.029205 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 09:08:46.029205 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:08:46.029205 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 09:08:46.029205 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Aug 24 09:08:46.031899 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 378/766] 10.222.0.16 () {56 vars in 1129 bytes} [Wed Aug 24 09:08:45 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 09:08:46.033393 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-02f14c67-eed4-43ed-94e6-01461b3dc2ca None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:08:46.033393 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:08:46.033393 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:08:46.033393 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:08:46.033393 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:08:46.033393 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:08:46.033393 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:08:46.033393 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:08:46.033393 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:08:46.033393 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:08:46.033393 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:08:46.033393 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:08:46.033393 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 09:08:46.033393 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 09:08:46.033393 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 09:08:46.034263 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 09:08:46.034263 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:08:46.034263 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:08:46.034263 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:08:46.034263 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:08:46.034263 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:08:46.034263 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:08:46.034263 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:08:46.034263 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 24 09:08:46.034263 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:08:46.034263 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:08:46.034263 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:08:46.034263 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:08:46.034263 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:08:46.034263 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:08:46.034263 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:08:46.034263 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:08:46.035149 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 09:08:46.035149 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 09:08:46.035149 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 09:08:46.035149 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 09:08:46.035149 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 09:08:46.035149 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 09:08:46.035149 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:08:46.035149 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 24 09:08:46.040119 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b5cb94-a143-4627-b2f5-7016761ba3e5 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:46.045991 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 389/767] 10.222.0.16 () {56 vars in 1111 bytes} [Wed Aug 24 09:08:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 09:08:46.047109 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f14c67-eed4-43ed-94e6-01461b3dc2ca None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:46.128559 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:46.191076 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b5cb94-a143-4627-b2f5-7016761ba3e5 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:46.227518 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f14c67-eed4-43ed-94e6-01461b3dc2ca None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:46.230136 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf60946c-12ab-4626-bfd6-0d879b789a33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:46.230870 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf60946c-12ab-4626-bfd6-0d879b789a33 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:46.231156 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf60946c-12ab-4626-bfd6-0d879b789a33 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:46.231426 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf60946c-12ab-4626-bfd6-0d879b789a33 None admin] PATH_INFO: `/v3/services` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:46.232207 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf60946c-12ab-4626-bfd6-0d879b789a33 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:46.232517 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf60946c-12ab-4626-bfd6-0d879b789a33 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:46.233991 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf60946c-12ab-4626-bfd6-0d879b789a33 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:46.239216 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 379/768] 10.222.0.16 () {56 vars in 1124 bytes} [Wed Aug 24 09:08:46 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:46.240267 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc848e6e-a2f8-4293-a35c-6836c9f8e5bb None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:46.240552 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc848e6e-a2f8-4293-a35c-6836c9f8e5bb None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:46.240827 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc848e6e-a2f8-4293-a35c-6836c9f8e5bb None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:46.247198 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 380/769] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:08:46 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:08:46.247934 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc848e6e-a2f8-4293-a35c-6836c9f8e5bb None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:46.272316 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07563f4-ae49-4566-837b-9dad5f08bd0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:46.273751 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07563f4-ae49-4566-837b-9dad5f08bd0a None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:46.274757 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07563f4-ae49-4566-837b-9dad5f08bd0a None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:46.275277 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07563f4-ae49-4566-837b-9dad5f08bd0a None admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:46.276773 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07563f4-ae49-4566-837b-9dad5f08bd0a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:46.277304 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07563f4-ae49-4566-837b-9dad5f08bd0a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:46.280111 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07563f4-ae49-4566-837b-9dad5f08bd0a None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:46.298837 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 390/770] 10.222.0.16 () {56 vars in 1148 bytes} [Wed Aug 24 09:08:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:08:46.306091 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2f1110-4bc6-4755-a209-943c23615ef3 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:46.306470 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2f1110-4bc6-4755-a209-943c23615ef3 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:46.306900 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2f1110-4bc6-4755-a209-943c23615ef3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:46.331886 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-de2f1110-4bc6-4755-a209-943c23615ef3 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:08:46.333327 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-de2f1110-4bc6-4755-a209-943c23615ef3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:46.344340 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:46.372433 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc848e6e-a2f8-4293-a35c-6836c9f8e5bb None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:46.390455 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:46.408968 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578cda97-e93f-461b-a322-d9355c8bbb5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:46.409693 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578cda97-e93f-461b-a322-d9355c8bbb5d None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:46.410068 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578cda97-e93f-461b-a322-d9355c8bbb5d None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:46.410563 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578cda97-e93f-461b-a322-d9355c8bbb5d None admin] PATH_INFO: `/v3/services` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:46.415604 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578cda97-e93f-461b-a322-d9355c8bbb5d None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:46.415604 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578cda97-e93f-461b-a322-d9355c8bbb5d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:46.415604 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578cda97-e93f-461b-a322-d9355c8bbb5d None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:46.416801 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 391/771] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:08:46 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 09:08:46.417853 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2f1110-4bc6-4755-a209-943c23615ef3 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:46.418986 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 381/772] 10.222.0.16 () {56 vars in 1124 bytes} [Wed Aug 24 09:08:46 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:46.425215 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578cda97-e93f-461b-a322-d9355c8bbb5d None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:46.527299 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578cda97-e93f-461b-a322-d9355c8bbb5d None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:46.528246 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2f1110-4bc6-4755-a209-943c23615ef3 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:46.562991 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ded4ad9-1594-49da-9177-57d7fa1e0aba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:46.563889 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ded4ad9-1594-49da-9177-57d7fa1e0aba None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:46.564265 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ded4ad9-1594-49da-9177-57d7fa1e0aba None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:46.564626 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ded4ad9-1594-49da-9177-57d7fa1e0aba None admin] PATH_INFO: `/v3/endpoints` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:46.565428 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ded4ad9-1594-49da-9177-57d7fa1e0aba None admin] RBAC: Adding query filter params (interface=public, service_id=b4d5ad4269fd4762aa8711bddac0121c, region_id=RegionOne) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:46.565851 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ded4ad9-1594-49da-9177-57d7fa1e0aba None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:46.567498 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ded4ad9-1594-49da-9177-57d7fa1e0aba None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:46.567695 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a80c1a-2eba-43e0-bbd5-aea88174b79a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:46.568507 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a80c1a-2eba-43e0-bbd5-aea88174b79a None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:46.568972 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a80c1a-2eba-43e0-bbd5-aea88174b79a None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:46.569523 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a80c1a-2eba-43e0-bbd5-aea88174b79a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:46.570433 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a80c1a-2eba-43e0-bbd5-aea88174b79a None admin] RBAC: Adding query filter params (role.id=67fd70a7d9794f7fb8faebffdec46dd2, scope.project.id=c467e88e2fb0491bacefcf6d2ce326d9, user.id=0489462287fc4aea8e8e720af24e1ef8) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:46.570920 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a80c1a-2eba-43e0-bbd5-aea88174b79a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:46.572571 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a80c1a-2eba-43e0-bbd5-aea88174b79a None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:46.577577 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 382/773] 10.222.0.16 () {56 vars in 1251 bytes} [Wed Aug 24 09:08:46 2022] GET /identity/v3/endpoints?service_id=b4d5ad4269fd4762aa8711bddac0121c&interface=public®ion_id=RegionOne => generated 194 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:08:46.579126 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 392/774] 10.222.0.16 () {56 vars in 1374 bytes} [Wed Aug 24 09:08:46 2022] GET /identity/v3/role_assignments?user.id=0489462287fc4aea8e8e720af24e1ef8&scope.project.id=c467e88e2fb0491bacefcf6d2ce326d9&role.id=67fd70a7d9794f7fb8faebffdec46dd2 => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:46.587700 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ded4ad9-1594-49da-9177-57d7fa1e0aba None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:46.640192 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:46.696980 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ded4ad9-1594-49da-9177-57d7fa1e0aba None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:46.739437 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4798f3-34b7-4741-9ee9-6b8646f8e0c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:46.740091 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4798f3-34b7-4741-9ee9-6b8646f8e0c6 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:46.740368 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4798f3-34b7-4741-9ee9-6b8646f8e0c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:46.740638 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4798f3-34b7-4741-9ee9-6b8646f8e0c6 None admin] PATH_INFO: `/v3/services` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:46.741250 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4798f3-34b7-4741-9ee9-6b8646f8e0c6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:46.742619 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4798f3-34b7-4741-9ee9-6b8646f8e0c6 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:46.755921 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 383/775] 10.222.0.16 () {56 vars in 1087 bytes} [Wed Aug 24 09:08:46 2022] GET /identity/v3/services => generated 772 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:08:46.869735 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a80c1a-2eba-43e0-bbd5-aea88174b79a None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:46.884101 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:46.995737 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a80c1a-2eba-43e0-bbd5-aea88174b79a None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:47.042109 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b440f3-5811-4782-b560-6a886a0e6b95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:47.042747 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b440f3-5811-4782-b560-6a886a0e6b95 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:47.043019 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b440f3-5811-4782-b560-6a886a0e6b95 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:47.043286 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b440f3-5811-4782-b560-6a886a0e6b95 None admin] PATH_INFO: `/v3/services/network` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:47.043900 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b440f3-5811-4782-b560-6a886a0e6b95 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:47.045258 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b440f3-5811-4782-b560-6a886a0e6b95 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:47.056404 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-54b440f3-5811-4782-b560-6a886a0e6b95 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 09:08:47.056404 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:08:47.056404 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 09:08:47.056404 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 09:08:47.056404 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 09:08:47.056404 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 09:08:47.056404 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 09:08:47.056404 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:08:47.056404 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 09:08:47.056404 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 24 09:08:47.056404 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 09:08:47.056404 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 24 09:08:47.056404 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:08:47.056404 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:08:47.056404 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:08:47.056404 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:08:47.056404 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:08:47.057455 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:08:47.057455 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:08:47.057455 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:08:47.057455 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:08:47.057455 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:08:47.057455 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:08:47.057455 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 09:08:47.057455 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 09:08:47.057455 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 09:08:47.057455 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 09:08:47.057455 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:08:47.057455 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:08:47.057455 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:08:47.057455 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:08:47.057455 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:08:47.057455 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:08:47.058322 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:08:47.058322 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 24 09:08:47.058322 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:08:47.058322 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:08:47.058322 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:08:47.058322 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:08:47.058322 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:08:47.058322 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:08:47.058322 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:08:47.058322 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:08:47.058322 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 09:08:47.058322 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:08:47.058322 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 09:08:47.058322 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 24 09:08:47.058322 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 393/776] 10.222.0.16 () {56 vars in 1111 bytes} [Wed Aug 24 09:08:46 2022] GET /identity/v3/services/network => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:08:47.063562 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4798f3-34b7-4741-9ee9-6b8646f8e0c6 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:47.172101 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002c3c56-d1e1-4490-bdb4-fa090733e558 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:47.172347 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002c3c56-d1e1-4490-bdb4-fa090733e558 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:47.172572 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002c3c56-d1e1-4490-bdb4-fa090733e558 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:47.174039 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 394/777] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:08:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:08:47.180429 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ceb7c69-29ee-40ed-aacf-10e5baef1297 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:47.180962 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ceb7c69-29ee-40ed-aacf-10e5baef1297 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:47.181421 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ceb7c69-29ee-40ed-aacf-10e5baef1297 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:47.189891 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4798f3-34b7-4741-9ee9-6b8646f8e0c6 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:47.206342 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-3ceb7c69-29ee-40ed-aacf-10e5baef1297 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:08:47.207996 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-3ceb7c69-29ee-40ed-aacf-10e5baef1297 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:47.215908 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:47.259271 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277734f8-e411-46c0-a35a-942282da0c1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:47.259980 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277734f8-e411-46c0-a35a-942282da0c1a None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:47.260265 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277734f8-e411-46c0-a35a-942282da0c1a None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:47.260542 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277734f8-e411-46c0-a35a-942282da0c1a None admin] PATH_INFO: `/v3/services` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:47.261188 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277734f8-e411-46c0-a35a-942282da0c1a None admin] RBAC: Adding query filter params (name=network) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:47.261491 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277734f8-e411-46c0-a35a-942282da0c1a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:47.262861 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277734f8-e411-46c0-a35a-942282da0c1a None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:47.276235 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 384/778] 10.222.0.16 () {56 vars in 1112 bytes} [Wed Aug 24 09:08:47 2022] GET /identity/v3/services?name=network => generated 124 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:47.281149 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277734f8-e411-46c0-a35a-942282da0c1a None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:47.294881 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 395/779] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:08:47 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 09:08:47.296878 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08d43f9-58b4-4c1d-a77c-34aaffa71481 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:47.297344 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08d43f9-58b4-4c1d-a77c-34aaffa71481 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:47.297841 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08d43f9-58b4-4c1d-a77c-34aaffa71481 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:47.299644 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 396/780] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:08:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:08:47.309028 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6413db-7d71-4ef1-a9cc-cfcbc3e722f3 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:47.309381 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6413db-7d71-4ef1-a9cc-cfcbc3e722f3 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:47.309777 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6413db-7d71-4ef1-a9cc-cfcbc3e722f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:47.318657 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:47.344782 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-5a6413db-7d71-4ef1-a9cc-cfcbc3e722f3 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:08:47.346314 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-5a6413db-7d71-4ef1-a9cc-cfcbc3e722f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:47.401352 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277734f8-e411-46c0-a35a-942282da0c1a None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:47.441582 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c5e81e-968f-475f-8f0a-f8b891d04145 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:47.442121 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c5e81e-968f-475f-8f0a-f8b891d04145 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:47.442402 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c5e81e-968f-475f-8f0a-f8b891d04145 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:47.442667 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c5e81e-968f-475f-8f0a-f8b891d04145 None admin] PATH_INFO: `/v3/services` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:47.443294 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c5e81e-968f-475f-8f0a-f8b891d04145 None admin] RBAC: Adding query filter params (type=network) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:47.443613 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c5e81e-968f-475f-8f0a-f8b891d04145 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:47.445148 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c5e81e-968f-475f-8f0a-f8b891d04145 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:47.455063 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 385/781] 10.222.0.16 () {56 vars in 1112 bytes} [Wed Aug 24 09:08:47 2022] GET /identity/v3/services?type=network => generated 350 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:08:47.460910 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c5e81e-968f-475f-8f0a-f8b891d04145 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:47.489090 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 397/782] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:08:47 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 09:08:47.515957 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:47.572512 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c5e81e-968f-475f-8f0a-f8b891d04145 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:47.608514 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a245a19d-3efe-499a-97e1-c72786b50dc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:47.609222 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a245a19d-3efe-499a-97e1-c72786b50dc3 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:47.609501 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a245a19d-3efe-499a-97e1-c72786b50dc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:47.609769 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a245a19d-3efe-499a-97e1-c72786b50dc3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:47.610464 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a245a19d-3efe-499a-97e1-c72786b50dc3 None admin] RBAC: Adding query filter params (interface=public, service_id=d4b0aaa09bed4a3c8502b7f9ecb8510b, region_id=RegionOne) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:47.610822 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a245a19d-3efe-499a-97e1-c72786b50dc3 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:47.612301 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a245a19d-3efe-499a-97e1-c72786b50dc3 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:47.617696 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 386/783] 10.222.0.16 () {56 vars in 1251 bytes} [Wed Aug 24 09:08:47 2022] GET /identity/v3/endpoints?service_id=d4b0aaa09bed4a3c8502b7f9ecb8510b&interface=public®ion_id=RegionOne => generated 194 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:47.629140 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6413db-7d71-4ef1-a9cc-cfcbc3e722f3 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:47.692103 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a245a19d-3efe-499a-97e1-c72786b50dc3 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:47.802108 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6413db-7d71-4ef1-a9cc-cfcbc3e722f3 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:47.849168 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a245a19d-3efe-499a-97e1-c72786b50dc3 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:47.851127 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:47.869000 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:47.881246 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd50632-456f-401e-8da4-1424c0d2e574 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:47.882032 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd50632-456f-401e-8da4-1424c0d2e574 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:47.882464 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd50632-456f-401e-8da4-1424c0d2e574 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:47.882834 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd50632-456f-401e-8da4-1424c0d2e574 None admin] PATH_INFO: `/v3/services` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:47.883570 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd50632-456f-401e-8da4-1424c0d2e574 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:47.885739 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd50632-456f-401e-8da4-1424c0d2e574 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:47.894857 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4d4a82-9aeb-4432-9d06-e769307e151c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:47.895495 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4d4a82-9aeb-4432-9d06-e769307e151c None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:47.895803 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4d4a82-9aeb-4432-9d06-e769307e151c None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:47.896074 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4d4a82-9aeb-4432-9d06-e769307e151c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:47.899147 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 398/784] 10.222.0.16 () {56 vars in 1087 bytes} [Wed Aug 24 09:08:47 2022] GET /identity/v3/services => generated 772 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:47.899876 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd50632-456f-401e-8da4-1424c0d2e574 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:47.921547 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4d4a82-9aeb-4432-9d06-e769307e151c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:08:47.929512 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-9a4d4a82-9aeb-4432-9d06-e769307e151c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:08:47.929512 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:08:47.929512 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:08:47.929512 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:08:47.929512 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:08:47.929512 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:08:47.929512 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:08:47.929512 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:08:47.929512 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:08:47.929512 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:08:47.929512 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:08:47.929512 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:08:47.929512 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:08:47.929512 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:08:47.929512 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:08:47.929512 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:08:47.930534 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:08:47.930534 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:08:47.930534 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:08:47.930534 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:08:47.930534 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:08:47.930534 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:08:47.930534 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:08:47.930534 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:08:47.930534 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:08:47.930534 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:08:47.930534 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:08:47.930534 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Aug 24 09:08:47.930534 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:08:47.930534 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:08:47.930534 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:08:47.930534 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:08:47.931389 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:08:47.931389 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:08:47.931389 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:08:47.931389 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:08:47.931389 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:08:47.931389 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:08:47.931389 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:08:47.931389 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:08:47.931389 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:08:47.931389 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:08:47.931389 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:08:47.931389 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Aug 24 09:08:47.931389 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 387/785] 10.222.0.16 () {56 vars in 1096 bytes} [Wed Aug 24 09:08:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 09:08:47.937667 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4d4a82-9aeb-4432-9d06-e769307e151c None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:48.051440 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4d4a82-9aeb-4432-9d06-e769307e151c None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:48.093231 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9928ffe6-f095-4c72-b859-1300ed61e543 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:48.094198 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9928ffe6-f095-4c72-b859-1300ed61e543 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:48.095196 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9928ffe6-f095-4c72-b859-1300ed61e543 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:48.095723 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9928ffe6-f095-4c72-b859-1300ed61e543 None admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:48.096595 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9928ffe6-f095-4c72-b859-1300ed61e543 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:48.097608 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9928ffe6-f095-4c72-b859-1300ed61e543 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:48.099892 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9928ffe6-f095-4c72-b859-1300ed61e543 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:48.112254 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 388/786] 10.222.0.16 () {56 vars in 1099 bytes} [Wed Aug 24 09:08:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:08:48.118130 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9928ffe6-f095-4c72-b859-1300ed61e543 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:48.132112 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd50632-456f-401e-8da4-1424c0d2e574 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:48.151970 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:48.169287 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a428ad5c-b125-41a5-aa1d-0e7b540a5f19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:48.170115 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a428ad5c-b125-41a5-aa1d-0e7b540a5f19 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:48.170475 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a428ad5c-b125-41a5-aa1d-0e7b540a5f19 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:48.170898 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a428ad5c-b125-41a5-aa1d-0e7b540a5f19 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:48.182524 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a428ad5c-b125-41a5-aa1d-0e7b540a5f19 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:08:48.188960 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-a428ad5c-b125-41a5-aa1d-0e7b540a5f19 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:08:48.188960 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:08:48.188960 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:08:48.188960 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:08:48.188960 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:08:48.188960 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:08:48.188960 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:08:48.188960 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:08:48.188960 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:08:48.188960 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:08:48.188960 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:08:48.188960 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:08:48.188960 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:08:48.188960 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:08:48.188960 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:08:48.188960 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:08:48.190252 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:08:48.190252 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:08:48.190252 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:08:48.190252 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:08:48.190252 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:08:48.190252 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:08:48.190252 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:08:48.190252 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:08:48.190252 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:08:48.190252 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:08:48.190252 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:08:48.190252 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 24 09:08:48.190252 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:08:48.190252 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:08:48.190252 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:08:48.190252 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:08:48.191303 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:08:48.191303 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:08:48.191303 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:08:48.191303 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:08:48.191303 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:08:48.191303 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:08:48.191303 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:08:48.191303 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:08:48.191303 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:08:48.191303 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:08:48.191303 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:08:48.191303 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 24 09:08:48.193086 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 399/787] 10.222.0.16 () {56 vars in 1096 bytes} [Wed Aug 24 09:08:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:08:48.200239 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a428ad5c-b125-41a5-aa1d-0e7b540a5f19 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:48.269678 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:48.272007 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9928ffe6-f095-4c72-b859-1300ed61e543 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:48.365490 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6a9eb9-ae8d-40cd-990a-e7ead1cda337 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:48.366449 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6a9eb9-ae8d-40cd-990a-e7ead1cda337 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:48.366882 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6a9eb9-ae8d-40cd-990a-e7ead1cda337 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:48.367394 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6a9eb9-ae8d-40cd-990a-e7ead1cda337 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:48.392537 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6a9eb9-ae8d-40cd-990a-e7ead1cda337 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:48.394308 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6a9eb9-ae8d-40cd-990a-e7ead1cda337 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:48.421554 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-6d6a9eb9-ae8d-40cd-990a-e7ead1cda337 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:08:48.421554 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:08:48.421554 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:08:48.421554 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:08:48.421554 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:08:48.421554 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:08:48.421554 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:08:48.421554 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:08:48.421554 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:08:48.421554 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:08:48.421554 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:08:48.421554 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:08:48.421554 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:08:48.421554 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:08:48.421554 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:08:48.421554 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:08:48.422757 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:08:48.422757 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:08:48.422757 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:08:48.422757 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:08:48.422757 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:08:48.422757 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:08:48.422757 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:08:48.422757 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Aug 24 09:08:48.422757 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:08:48.422757 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:08:48.422757 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:08:48.422757 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:08:48.422757 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:08:48.422757 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:08:48.422757 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:08:48.422757 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:08:48.422757 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:08:48.423898 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:08:48.423898 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:08:48.423898 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Aug 24 09:08:48.431430 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 389/788] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 24 09:08:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:08:48.448596 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6a9eb9-ae8d-40cd-990a-e7ead1cda337 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:48.457286 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a428ad5c-b125-41a5-aa1d-0e7b540a5f19 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:48.496233 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59aa9f9-004e-4174-aa10-dac8f3d8905d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:48.496997 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59aa9f9-004e-4174-aa10-dac8f3d8905d None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:48.497402 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59aa9f9-004e-4174-aa10-dac8f3d8905d None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:48.498060 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59aa9f9-004e-4174-aa10-dac8f3d8905d None admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:48.498804 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59aa9f9-004e-4174-aa10-dac8f3d8905d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:48.499270 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59aa9f9-004e-4174-aa10-dac8f3d8905d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:48.500777 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59aa9f9-004e-4174-aa10-dac8f3d8905d None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:48.510718 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 400/789] 10.222.0.16 () {56 vars in 1099 bytes} [Wed Aug 24 09:08:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:48.518065 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59aa9f9-004e-4174-aa10-dac8f3d8905d None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:48.609933 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:48.618499 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6a9eb9-ae8d-40cd-990a-e7ead1cda337 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:48.654917 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6da6bf-24c6-4676-b489-ce7cb83c532c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:48.655962 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6da6bf-24c6-4676-b489-ce7cb83c532c None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:48.656242 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6da6bf-24c6-4676-b489-ce7cb83c532c None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:48.656521 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6da6bf-24c6-4676-b489-ce7cb83c532c None admin] PATH_INFO: `/v3/domains` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:48.657168 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6da6bf-24c6-4676-b489-ce7cb83c532c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:48.657476 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6da6bf-24c6-4676-b489-ce7cb83c532c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:48.658854 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6da6bf-24c6-4676-b489-ce7cb83c532c None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:48.673719 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 390/790] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Aug 24 09:08:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:48.680883 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6da6bf-24c6-4676-b489-ce7cb83c532c None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:48.730889 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59aa9f9-004e-4174-aa10-dac8f3d8905d None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:48.750773 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:48.770432 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d12903-454f-4e59-b0c6-3aaf154d9191 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:48.771197 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d12903-454f-4e59-b0c6-3aaf154d9191 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:48.771481 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d12903-454f-4e59-b0c6-3aaf154d9191 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:48.771785 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d12903-454f-4e59-b0c6-3aaf154d9191 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:48.784163 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d12903-454f-4e59-b0c6-3aaf154d9191 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:48.786068 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d12903-454f-4e59-b0c6-3aaf154d9191 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:48.802589 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-77d12903-454f-4e59-b0c6-3aaf154d9191 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:08:48.802589 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:08:48.802589 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:08:48.802589 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:08:48.802589 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:08:48.802589 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:08:48.802589 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:08:48.802589 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:08:48.802589 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:08:48.802589 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:08:48.802589 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:08:48.802589 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:08:48.802589 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:08:48.802589 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:08:48.802589 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:08:48.802589 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:08:48.803623 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:08:48.803623 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:08:48.803623 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:08:48.803623 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:08:48.803623 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:08:48.803623 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:08:48.803623 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:08:48.803623 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 24 09:08:48.803623 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:08:48.803623 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:08:48.803623 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:08:48.803623 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:08:48.803623 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:08:48.803623 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:08:48.803623 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:08:48.803623 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:08:48.803623 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:08:48.804550 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:08:48.804550 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:08:48.804550 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 24 09:08:48.808312 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 401/791] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 24 09:08:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:08:48.814128 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d12903-454f-4e59-b0c6-3aaf154d9191 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:48.864576 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6da6bf-24c6-4676-b489-ce7cb83c532c None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:48.903931 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2798056-79b4-4feb-b13a-6fa5cfaa5762 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:48.906875 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2798056-79b4-4feb-b13a-6fa5cfaa5762 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:48.907367 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2798056-79b4-4feb-b13a-6fa5cfaa5762 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:48.908755 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2798056-79b4-4feb-b13a-6fa5cfaa5762 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:48.915944 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:48.921438 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2798056-79b4-4feb-b13a-6fa5cfaa5762 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:48.923208 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2798056-79b4-4feb-b13a-6fa5cfaa5762 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:48.936391 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-d2798056-79b4-4feb-b13a-6fa5cfaa5762 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 09:08:48.936391 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:08:48.936391 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:08:48.936391 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:08:48.936391 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:08:48.936391 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:08:48.936391 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:08:48.936391 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:08:48.936391 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:08:48.936391 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:08:48.936391 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:08:48.936391 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:08:48.936391 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 09:08:48.936391 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 09:08:48.936391 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 09:08:48.936391 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 09:08:48.937395 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:08:48.937395 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:08:48.937395 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 09:08:48.937395 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:08:48.937395 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 09:08:48.937395 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:08:48.937395 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:08:48.937395 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:08:48.937395 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:08:48.937395 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:08:48.937395 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:08:48.937395 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Aug 24 09:08:48.937395 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:08:48.937395 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:08:48.937395 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:08:48.937395 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:08:48.937395 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:08:48.937395 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:08:48.938360 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:08:48.938360 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:08:48.938360 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 09:08:48.938360 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 09:08:48.938360 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 09:08:48.938360 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 09:08:48.938360 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 09:08:48.938360 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 09:08:48.938360 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 09:08:48.938360 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Aug 24 09:08:48.942267 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 391/792] 10.222.0.16 () {56 vars in 1096 bytes} [Wed Aug 24 09:08:48 2022] GET /identity/v3/users/swift => generated 83 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:08:48.943571 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd17f61-c66f-4f0a-bbe5-5b02069ba41b None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:48.943878 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd17f61-c66f-4f0a-bbe5-5b02069ba41b None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:48.944334 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd17f61-c66f-4f0a-bbe5-5b02069ba41b None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:48.949755 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 392/793] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:08:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:08:48.949755 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd17f61-c66f-4f0a-bbe5-5b02069ba41b None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:49.012102 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d12903-454f-4e59-b0c6-3aaf154d9191 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:49.057624 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a0f311-1dbc-46f1-8cd4-1fa75f824f91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:49.058234 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a0f311-1dbc-46f1-8cd4-1fa75f824f91 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:49.058513 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a0f311-1dbc-46f1-8cd4-1fa75f824f91 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:49.058781 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a0f311-1dbc-46f1-8cd4-1fa75f824f91 None admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:49.059418 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a0f311-1dbc-46f1-8cd4-1fa75f824f91 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:49.059761 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a0f311-1dbc-46f1-8cd4-1fa75f824f91 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:49.061332 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a0f311-1dbc-46f1-8cd4-1fa75f824f91 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:49.061470 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd17f61-c66f-4f0a-bbe5-5b02069ba41b None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:49.073616 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 402/794] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Aug 24 09:08:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:49.074858 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7aeb235-c69b-4d29-9cd2-8e77500807d1 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:49.075137 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7aeb235-c69b-4d29-9cd2-8e77500807d1 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:49.075406 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7aeb235-c69b-4d29-9cd2-8e77500807d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:49.093885 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-d7aeb235-c69b-4d29-9cd2-8e77500807d1 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:08:49.095062 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-d7aeb235-c69b-4d29-9cd2-8e77500807d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:49.104448 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ddb033-3271-4af9-95be-933b82cb47cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:49.105134 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ddb033-3271-4af9-95be-933b82cb47cf None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:49.105479 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ddb033-3271-4af9-95be-933b82cb47cf None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:49.106330 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ddb033-3271-4af9-95be-933b82cb47cf None admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:49.107144 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ddb033-3271-4af9-95be-933b82cb47cf None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:49.107918 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ddb033-3271-4af9-95be-933b82cb47cf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:49.109706 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ddb033-3271-4af9-95be-933b82cb47cf None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:49.122177 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:49.134956 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 393/795] 10.222.0.16 () {56 vars in 1135 bytes} [Wed Aug 24 09:08:48 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:49.136626 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ddb033-3271-4af9-95be-933b82cb47cf None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:49.183101 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 403/796] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:08:49 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 09:08:49.184281 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7aeb235-c69b-4d29-9cd2-8e77500807d1 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:49.234512 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:49.287388 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ddb033-3271-4af9-95be-933b82cb47cf None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:49.314213 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7aeb235-c69b-4d29-9cd2-8e77500807d1 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:49.345097 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae8ce7d-d48d-4049-8608-056aeee0efe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:49.346002 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae8ce7d-d48d-4049-8608-056aeee0efe1 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:49.346388 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae8ce7d-d48d-4049-8608-056aeee0efe1 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:49.346763 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae8ce7d-d48d-4049-8608-056aeee0efe1 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:49.352075 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4192053-61c2-4724-ba22-231b9f69282e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:49.352891 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4192053-61c2-4724-ba22-231b9f69282e None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:49.353318 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4192053-61c2-4724-ba22-231b9f69282e None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:49.354300 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4192053-61c2-4724-ba22-231b9f69282e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:49.356052 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae8ce7d-d48d-4049-8608-056aeee0efe1 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:49.357860 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae8ce7d-d48d-4049-8608-056aeee0efe1 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:49.365845 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4192053-61c2-4724-ba22-231b9f69282e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:49.367602 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4192053-61c2-4724-ba22-231b9f69282e None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:49.372013 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-7ae8ce7d-d48d-4049-8608-056aeee0efe1 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 09:08:49.372013 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:08:49.372013 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:08:49.372013 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:08:49.372013 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:08:49.372013 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:08:49.372013 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:08:49.372013 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:08:49.372013 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:08:49.372013 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:08:49.372013 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:08:49.372013 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:08:49.372013 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 09:08:49.372013 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 09:08:49.372013 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 09:08:49.372013 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 09:08:49.373191 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:08:49.373191 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:08:49.373191 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 09:08:49.373191 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:08:49.373191 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 09:08:49.373191 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:08:49.373191 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:08:49.373191 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:08:49.373191 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:08:49.373191 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:08:49.373191 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:08:49.373191 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Aug 24 09:08:49.373191 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:08:49.373191 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:08:49.373191 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:08:49.373191 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:08:49.373191 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:08:49.373191 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:08:49.374173 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:08:49.374173 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:08:49.374173 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 09:08:49.374173 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 09:08:49.374173 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 09:08:49.374173 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 09:08:49.374173 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 09:08:49.374173 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 09:08:49.374173 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 09:08:49.374173 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Aug 24 09:08:49.374173 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 394/797] 10.222.0.16 () {56 vars in 1093 bytes} [Wed Aug 24 09:08:49 2022] GET /identity/v3/users/nova => generated 82 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:08:49.375642 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d417ba4-0bfd-4cc5-9873-7f72690553ae None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:49.376002 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d417ba4-0bfd-4cc5-9873-7f72690553ae None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:49.376346 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d417ba4-0bfd-4cc5-9873-7f72690553ae None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:49.382179 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 395/798] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:08:49 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:08:49.383108 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d417ba4-0bfd-4cc5-9873-7f72690553ae None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:49.388761 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-a4192053-61c2-4724-ba22-231b9f69282e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:08:49.388761 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:08:49.388761 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:08:49.388761 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:08:49.388761 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:08:49.388761 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:08:49.388761 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:08:49.388761 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:08:49.388761 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:08:49.388761 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:08:49.388761 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:08:49.388761 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:08:49.388761 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:08:49.388761 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:08:49.388761 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:08:49.388761 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:08:49.389716 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:08:49.389716 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:08:49.389716 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:08:49.389716 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:08:49.389716 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:08:49.389716 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:08:49.389716 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:08:49.389716 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 24 09:08:49.389716 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:08:49.389716 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:08:49.389716 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:08:49.389716 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:08:49.389716 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:08:49.389716 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:08:49.389716 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:08:49.389716 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:08:49.389716 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:08:49.390615 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:08:49.390615 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:08:49.390615 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 24 09:08:49.392844 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 404/799] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 24 09:08:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:08:49.394247 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8d6957-928e-40a4-bc7c-510c489f49d3 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:49.394706 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8d6957-928e-40a4-bc7c-510c489f49d3 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:49.395155 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8d6957-928e-40a4-bc7c-510c489f49d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:49.413593 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-2a8d6957-928e-40a4-bc7c-510c489f49d3 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:08:49.415083 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-2a8d6957-928e-40a4-bc7c-510c489f49d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:49.423308 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:49.489794 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 405/800] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:08:49 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 09:08:49.490651 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8d6957-928e-40a4-bc7c-510c489f49d3 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:49.499621 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d417ba4-0bfd-4cc5-9873-7f72690553ae None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:49.514340 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:49.535908 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3bc868-7ff1-40fd-aeb3-d6ea309bb913 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:49.536542 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3bc868-7ff1-40fd-aeb3-d6ea309bb913 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:49.536820 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3bc868-7ff1-40fd-aeb3-d6ea309bb913 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:49.537092 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3bc868-7ff1-40fd-aeb3-d6ea309bb913 None admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:49.537817 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3bc868-7ff1-40fd-aeb3-d6ea309bb913 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:49.538119 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3bc868-7ff1-40fd-aeb3-d6ea309bb913 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:49.539531 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3bc868-7ff1-40fd-aeb3-d6ea309bb913 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:49.561038 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 396/801] 10.222.0.16 () {56 vars in 1133 bytes} [Wed Aug 24 09:08:49 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:49.561980 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3bc868-7ff1-40fd-aeb3-d6ea309bb913 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:49.626353 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8d6957-928e-40a4-bc7c-510c489f49d3 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:49.673349 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f01033-6554-4f25-ab93-7dd846ad20a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:49.673963 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f01033-6554-4f25-ab93-7dd846ad20a3 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:49.674246 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f01033-6554-4f25-ab93-7dd846ad20a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:49.674528 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f01033-6554-4f25-ab93-7dd846ad20a3 None admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:49.675166 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f01033-6554-4f25-ab93-7dd846ad20a3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:49.675605 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f01033-6554-4f25-ab93-7dd846ad20a3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:49.677368 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f01033-6554-4f25-ab93-7dd846ad20a3 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:49.680970 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3bc868-7ff1-40fd-aeb3-d6ea309bb913 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:49.692572 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 406/802] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Aug 24 09:08:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:49.693896 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f01033-6554-4f25-ab93-7dd846ad20a3 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:49.697163 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:49.726900 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686b49f7-d14f-469a-8f9d-d96d1174ed73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:49.727536 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686b49f7-d14f-469a-8f9d-d96d1174ed73 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:49.727846 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686b49f7-d14f-469a-8f9d-d96d1174ed73 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:49.728119 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686b49f7-d14f-469a-8f9d-d96d1174ed73 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:49.745466 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686b49f7-d14f-469a-8f9d-d96d1174ed73 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:08:49.751984 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-686b49f7-d14f-469a-8f9d-d96d1174ed73 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 09:08:49.751984 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:08:49.751984 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:08:49.751984 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:08:49.751984 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:08:49.751984 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:08:49.751984 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:08:49.751984 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:08:49.751984 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:08:49.751984 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:08:49.751984 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:08:49.751984 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:08:49.751984 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:08:49.751984 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:08:49.751984 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:08:49.751984 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:08:49.752939 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:08:49.752939 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:08:49.752939 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:08:49.752939 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:08:49.752939 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:08:49.752939 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:08:49.752939 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:08:49.752939 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:08:49.752939 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:08:49.752939 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:08:49.752939 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:08:49.752939 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Aug 24 09:08:49.752939 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:08:49.752939 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:08:49.752939 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:08:49.752939 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:08:49.753782 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:08:49.753782 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:08:49.753782 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:08:49.753782 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:08:49.753782 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:08:49.753782 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:08:49.753782 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:08:49.753782 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:08:49.753782 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:08:49.753782 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:08:49.753782 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 09:08:49.753782 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Aug 24 09:08:49.757645 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 397/803] 10.222.0.16 () {56 vars in 1102 bytes} [Wed Aug 24 09:08:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:08:49.759191 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686b49f7-d14f-469a-8f9d-d96d1174ed73 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:49.801857 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:49.820654 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f01033-6554-4f25-ab93-7dd846ad20a3 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:49.857817 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0910b9c-fe23-41ab-93be-3b54a3c0933f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:49.858778 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0910b9c-fe23-41ab-93be-3b54a3c0933f None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:49.859268 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0910b9c-fe23-41ab-93be-3b54a3c0933f None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:49.859630 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0910b9c-fe23-41ab-93be-3b54a3c0933f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:49.871225 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686b49f7-d14f-469a-8f9d-d96d1174ed73 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:49.906787 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79496d1-58b2-4231-ad7c-c338f475d3e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:49.907694 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79496d1-58b2-4231-ad7c-c338f475d3e9 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:49.908130 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79496d1-58b2-4231-ad7c-c338f475d3e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:49.908398 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79496d1-58b2-4231-ad7c-c338f475d3e9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:49.915465 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79496d1-58b2-4231-ad7c-c338f475d3e9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:49.917499 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79496d1-58b2-4231-ad7c-c338f475d3e9 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:49.928424 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-c79496d1-58b2-4231-ad7c-c338f475d3e9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:08:49.928424 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:08:49.928424 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:08:49.928424 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:08:49.928424 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:08:49.928424 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:08:49.928424 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:08:49.928424 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:08:49.928424 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:08:49.928424 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:08:49.928424 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:08:49.928424 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:08:49.928424 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 09:08:49.928424 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 09:08:49.928424 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 09:08:49.929497 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 09:08:49.929497 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:08:49.929497 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:08:49.929497 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:08:49.929497 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:08:49.929497 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:08:49.929497 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:08:49.929497 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:08:49.929497 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Aug 24 09:08:49.929497 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:08:49.929497 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:08:49.929497 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:08:49.929497 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:08:49.929497 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:08:49.929497 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:08:49.929497 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:08:49.929497 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:08:49.930912 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 09:08:49.930912 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 09:08:49.930912 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 09:08:49.930912 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 09:08:49.930912 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 09:08:49.930912 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 09:08:49.930912 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:08:49.930912 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Aug 24 09:08:49.930912 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 398/804] 10.222.0.16 () {56 vars in 1111 bytes} [Wed Aug 24 09:08:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:08:49.932398 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79496d1-58b2-4231-ad7c-c338f475d3e9 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:49.934895 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0910b9c-fe23-41ab-93be-3b54a3c0933f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:49.936428 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0910b9c-fe23-41ab-93be-3b54a3c0933f None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:49.938504 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:50.025781 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:50.034491 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79496d1-58b2-4231-ad7c-c338f475d3e9 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:50.043468 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 407/805] 10.222.0.16 () {58 vars in 1282 bytes} [Wed Aug 24 09:08:49 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 09:08:50.043468 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0910b9c-fe23-41ab-93be-3b54a3c0933f None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:50.072050 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0ba510-fde9-48c9-80bf-0cee928aa3f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:50.072689 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0ba510-fde9-48c9-80bf-0cee928aa3f9 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:50.072976 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0ba510-fde9-48c9-80bf-0cee928aa3f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:50.073238 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0ba510-fde9-48c9-80bf-0cee928aa3f9 None admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:50.074013 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0ba510-fde9-48c9-80bf-0cee928aa3f9 None admin] RBAC: Adding query filter params (name=service) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:50.074315 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0ba510-fde9-48c9-80bf-0cee928aa3f9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:50.075828 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0ba510-fde9-48c9-80bf-0cee928aa3f9 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:50.090141 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 399/806] 10.222.0.16 () {56 vars in 1103 bytes} [Wed Aug 24 09:08:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:50.090141 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0ba510-fde9-48c9-80bf-0cee928aa3f9 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:50.151593 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0910b9c-fe23-41ab-93be-3b54a3c0933f None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:50.155742 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:50.186228 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84904898-5ec1-489b-8243-7281aee28195 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:50.186773 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84904898-5ec1-489b-8243-7281aee28195 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:50.187048 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84904898-5ec1-489b-8243-7281aee28195 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:50.187315 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84904898-5ec1-489b-8243-7281aee28195 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:50.187924 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84904898-5ec1-489b-8243-7281aee28195 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:50.189319 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84904898-5ec1-489b-8243-7281aee28195 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:50.194104 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0ba510-fde9-48c9-80bf-0cee928aa3f9 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:50.198529 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-84904898-5ec1-489b-8243-7281aee28195 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 09:08:50.198529 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:08:50.198529 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 09:08:50.198529 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 09:08:50.198529 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 09:08:50.198529 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 09:08:50.198529 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 09:08:50.198529 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:08:50.198529 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 09:08:50.198529 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 24 09:08:50.198529 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 09:08:50.198529 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 24 09:08:50.198529 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:08:50.198529 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:08:50.198529 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:08:50.198529 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:08:50.198529 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:08:50.199542 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:08:50.199542 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:08:50.199542 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:08:50.199542 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:08:50.199542 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:08:50.199542 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:08:50.199542 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 09:08:50.199542 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 09:08:50.199542 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 09:08:50.199542 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 09:08:50.199542 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:08:50.199542 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:08:50.199542 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:08:50.199542 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:08:50.199542 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:08:50.199542 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:08:50.205957 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:08:50.205957 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 24 09:08:50.205957 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:08:50.205957 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:08:50.205957 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:08:50.205957 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:08:50.205957 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:08:50.205957 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:08:50.205957 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:08:50.205957 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:08:50.205957 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 09:08:50.205957 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:08:50.205957 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 09:08:50.205957 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 24 09:08:50.205957 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 408/807] 10.222.0.16 () {56 vars in 1129 bytes} [Wed Aug 24 09:08:50 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:08:50.207262 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84904898-5ec1-489b-8243-7281aee28195 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:50.232607 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab7ec4d-38de-414a-abac-234d52abc9c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:50.233211 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab7ec4d-38de-414a-abac-234d52abc9c3 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:50.233489 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab7ec4d-38de-414a-abac-234d52abc9c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:50.233769 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab7ec4d-38de-414a-abac-234d52abc9c3 None admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:50.234495 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab7ec4d-38de-414a-abac-234d52abc9c3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:50.234801 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab7ec4d-38de-414a-abac-234d52abc9c3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:50.236262 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab7ec4d-38de-414a-abac-234d52abc9c3 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:50.248229 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 400/808] 10.222.0.16 () {56 vars in 1148 bytes} [Wed Aug 24 09:08:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:50.249076 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab7ec4d-38de-414a-abac-234d52abc9c3 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:50.265175 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:50.306262 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84904898-5ec1-489b-8243-7281aee28195 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:50.342627 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eaa3820-c1c6-4274-8c43-3ca334c44519 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:50.343262 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaa3820-c1c6-4274-8c43-3ca334c44519 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:50.343541 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaa3820-c1c6-4274-8c43-3ca334c44519 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:50.343844 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaa3820-c1c6-4274-8c43-3ca334c44519 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:50.349986 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab7ec4d-38de-414a-abac-234d52abc9c3 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:50.354634 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eaa3820-c1c6-4274-8c43-3ca334c44519 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:50.356094 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eaa3820-c1c6-4274-8c43-3ca334c44519 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:50.369911 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-9eaa3820-c1c6-4274-8c43-3ca334c44519 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:08:50.369911 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:08:50.369911 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:08:50.369911 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:08:50.369911 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:08:50.369911 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:08:50.369911 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:08:50.369911 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:08:50.369911 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:08:50.369911 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:08:50.369911 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:08:50.369911 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:08:50.369911 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:08:50.369911 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:08:50.369911 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:08:50.369911 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:08:50.371137 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:08:50.371137 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:08:50.371137 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:08:50.371137 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:08:50.371137 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:08:50.371137 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:08:50.371137 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:08:50.371137 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 24 09:08:50.371137 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:08:50.371137 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:08:50.371137 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:08:50.371137 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:08:50.371137 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:08:50.371137 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:08:50.371137 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:08:50.371137 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:08:50.371137 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:08:50.372062 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:08:50.372062 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:08:50.372062 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 24 09:08:50.372062 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 409/809] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 24 09:08:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:08:50.372062 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eaa3820-c1c6-4274-8c43-3ca334c44519 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:50.387553 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672018cf-54ec-4489-94f4-3f6b91eb0deb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:50.388630 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672018cf-54ec-4489-94f4-3f6b91eb0deb None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:50.389055 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672018cf-54ec-4489-94f4-3f6b91eb0deb None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:50.389328 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672018cf-54ec-4489-94f4-3f6b91eb0deb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:50.391081 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:50.399420 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672018cf-54ec-4489-94f4-3f6b91eb0deb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:50.401482 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672018cf-54ec-4489-94f4-3f6b91eb0deb None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:50.415692 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-672018cf-54ec-4489-94f4-3f6b91eb0deb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:08:50.415692 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:08:50.415692 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:08:50.415692 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:08:50.415692 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:08:50.415692 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:08:50.415692 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:08:50.415692 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:08:50.415692 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:08:50.415692 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:08:50.415692 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:08:50.415692 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:08:50.415692 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:08:50.415692 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:08:50.415692 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:08:50.415692 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:08:50.417206 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:08:50.417206 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:08:50.417206 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:08:50.417206 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:08:50.417206 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:08:50.417206 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:08:50.417206 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:08:50.417206 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Aug 24 09:08:50.417206 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:08:50.417206 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:08:50.417206 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:08:50.417206 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:08:50.417206 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:08:50.417206 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:08:50.417206 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:08:50.417206 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:08:50.417206 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:08:50.418109 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:08:50.418109 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:08:50.418109 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Aug 24 09:08:50.418109 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 401/810] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 24 09:08:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:08:50.418644 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccd7d36-1280-42c7-9dd1-9021d5a0480e None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:50.418935 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccd7d36-1280-42c7-9dd1-9021d5a0480e None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:50.419190 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccd7d36-1280-42c7-9dd1-9021d5a0480e None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:50.420572 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 402/811] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:08:50 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:08:50.421152 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccd7d36-1280-42c7-9dd1-9021d5a0480e None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:50.481469 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eaa3820-c1c6-4274-8c43-3ca334c44519 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:50.499951 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:50.522362 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6ff241-af8a-43a9-9c6e-d63c5845ebff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:50.523381 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6ff241-af8a-43a9-9c6e-d63c5845ebff None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:50.523846 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6ff241-af8a-43a9-9c6e-d63c5845ebff None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:50.526004 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6ff241-af8a-43a9-9c6e-d63c5845ebff None admin] PATH_INFO: `/v3/services` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:50.526935 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6ff241-af8a-43a9-9c6e-d63c5845ebff None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:50.527247 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6ff241-af8a-43a9-9c6e-d63c5845ebff None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:50.528635 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6ff241-af8a-43a9-9c6e-d63c5845ebff None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:50.533129 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 410/812] 10.222.0.16 () {56 vars in 1124 bytes} [Wed Aug 24 09:08:50 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:50.533921 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6ff241-af8a-43a9-9c6e-d63c5845ebff None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:50.542645 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccd7d36-1280-42c7-9dd1-9021d5a0480e None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:50.576891 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7904a636-ffb7-4b37-826a-d0f055ee9cf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:50.577548 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7904a636-ffb7-4b37-826a-d0f055ee9cf1 None admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:50.577826 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7904a636-ffb7-4b37-826a-d0f055ee9cf1 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:50.578101 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7904a636-ffb7-4b37-826a-d0f055ee9cf1 None admin] PATH_INFO: `/v3/projects/c467e88e2fb0491bacefcf6d2ce326d9/users/fdd5f0d67fe54e9fae6f4af644988d93/roles/15f634d4a2d247ddab90c458e5e1c141` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:50.580243 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7904a636-ffb7-4b37-826a-d0f055ee9cf1 None admin] RBAC: Authorizing `identity:create_grant(project_id=c467e88e2fb0491bacefcf6d2ce326d9, user_id=fdd5f0d67fe54e9fae6f4af644988d93, role_id=15f634d4a2d247ddab90c458e5e1c141)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:50.581654 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7904a636-ffb7-4b37-826a-d0f055ee9cf1 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:50.614598 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:50.631750 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6ff241-af8a-43a9-9c6e-d63c5845ebff None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:50.649631 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 403/813] 10.222.0.16 () {58 vars in 1439 bytes} [Wed Aug 24 09:08:50 2022] PUT /identity/v3/projects/c467e88e2fb0491bacefcf6d2ce326d9/users/fdd5f0d67fe54e9fae6f4af644988d93/roles/15f634d4a2d247ddab90c458e5e1c141 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:08:50.650977 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7904a636-ffb7-4b37-826a-d0f055ee9cf1 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:50.668329 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485665cd-da38-4842-8c2a-f1833a6ad77a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:50.668952 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485665cd-da38-4842-8c2a-f1833a6ad77a None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:50.669228 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485665cd-da38-4842-8c2a-f1833a6ad77a None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:50.669499 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485665cd-da38-4842-8c2a-f1833a6ad77a None admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:50.670152 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485665cd-da38-4842-8c2a-f1833a6ad77a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:50.670455 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485665cd-da38-4842-8c2a-f1833a6ad77a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:50.671948 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485665cd-da38-4842-8c2a-f1833a6ad77a None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:50.691548 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 411/814] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Aug 24 09:08:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:50.691548 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee81d15-236d-4b8f-8ec1-c625110c3ba8 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:50.691548 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee81d15-236d-4b8f-8ec1-c625110c3ba8 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:50.691548 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee81d15-236d-4b8f-8ec1-c625110c3ba8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:50.708175 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-bee81d15-236d-4b8f-8ec1-c625110c3ba8 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:08:50.709401 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-bee81d15-236d-4b8f-8ec1-c625110c3ba8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:50.784422 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 412/815] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:08:50 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 09:08:50.785274 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee81d15-236d-4b8f-8ec1-c625110c3ba8 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:50.801557 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7904a636-ffb7-4b37-826a-d0f055ee9cf1 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:50.837478 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:50.859330 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16749ce6-0ba2-4c65-9661-d0520acefffd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:50.860221 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16749ce6-0ba2-4c65-9661-d0520acefffd None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:50.860500 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16749ce6-0ba2-4c65-9661-d0520acefffd None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:50.861169 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16749ce6-0ba2-4c65-9661-d0520acefffd None admin] PATH_INFO: `/v3/domains` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:50.862124 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16749ce6-0ba2-4c65-9661-d0520acefffd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:50.862427 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16749ce6-0ba2-4c65-9661-d0520acefffd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:50.863891 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16749ce6-0ba2-4c65-9661-d0520acefffd None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:50.870165 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:50.881956 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 404/816] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Aug 24 09:08:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:50.886449 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16749ce6-0ba2-4c65-9661-d0520acefffd None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:50.911608 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee81d15-236d-4b8f-8ec1-c625110c3ba8 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:50.957147 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c7d246-ab87-4115-9de1-708856bc1e8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:50.957898 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c7d246-ab87-4115-9de1-708856bc1e8a None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:50.958257 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c7d246-ab87-4115-9de1-708856bc1e8a None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:50.958603 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c7d246-ab87-4115-9de1-708856bc1e8a None admin] PATH_INFO: `/v3/services` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:50.959315 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c7d246-ab87-4115-9de1-708856bc1e8a None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:50.959726 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c7d246-ab87-4115-9de1-708856bc1e8a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:50.961846 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c7d246-ab87-4115-9de1-708856bc1e8a None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:50.969387 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 413/817] 10.222.0.16 () {56 vars in 1124 bytes} [Wed Aug 24 09:08:50 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:50.970296 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c7d246-ab87-4115-9de1-708856bc1e8a None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:51.003670 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16749ce6-0ba2-4c65-9661-d0520acefffd None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:51.051610 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873241bd-4a2e-4142-8355-a384d64f5e73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:51.053020 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873241bd-4a2e-4142-8355-a384d64f5e73 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:51.053566 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873241bd-4a2e-4142-8355-a384d64f5e73 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:51.053988 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873241bd-4a2e-4142-8355-a384d64f5e73 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:51.063918 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873241bd-4a2e-4142-8355-a384d64f5e73 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:51.065374 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873241bd-4a2e-4142-8355-a384d64f5e73 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:51.079291 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-873241bd-4a2e-4142-8355-a384d64f5e73 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:08:51.079291 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:08:51.079291 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:08:51.079291 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:08:51.079291 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:08:51.079291 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:08:51.079291 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:08:51.079291 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:08:51.079291 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:08:51.079291 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:08:51.079291 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:08:51.079291 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:08:51.079291 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 09:08:51.079291 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 09:08:51.079291 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 09:08:51.080269 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 09:08:51.080269 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:08:51.080269 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:08:51.080269 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:08:51.080269 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:08:51.080269 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:08:51.080269 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:08:51.080269 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:08:51.080269 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Aug 24 09:08:51.080269 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:08:51.080269 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:08:51.080269 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:08:51.080269 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:08:51.080269 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:08:51.080269 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:08:51.080269 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:08:51.080269 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:08:51.081172 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 09:08:51.081172 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 09:08:51.081172 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 09:08:51.081172 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 09:08:51.081172 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 09:08:51.081172 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 09:08:51.081172 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:08:51.081172 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Aug 24 09:08:51.081172 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 405/818] 10.222.0.16 () {56 vars in 1111 bytes} [Wed Aug 24 09:08:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:08:51.081662 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873241bd-4a2e-4142-8355-a384d64f5e73 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:51.102114 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c7d246-ab87-4115-9de1-708856bc1e8a None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:51.133343 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:51.136560 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8123040-3f2f-47b0-833f-8bd1d3d1bdda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:51.137196 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8123040-3f2f-47b0-833f-8bd1d3d1bdda None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:51.137474 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8123040-3f2f-47b0-833f-8bd1d3d1bdda None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:51.138059 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8123040-3f2f-47b0-833f-8bd1d3d1bdda None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:51.140975 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:51.146739 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8123040-3f2f-47b0-833f-8bd1d3d1bdda None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:51.148288 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8123040-3f2f-47b0-833f-8bd1d3d1bdda None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:51.159568 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-b8123040-3f2f-47b0-833f-8bd1d3d1bdda None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 09:08:51.159568 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:08:51.159568 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:08:51.159568 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:08:51.159568 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:08:51.159568 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:08:51.159568 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:08:51.159568 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:08:51.159568 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:08:51.159568 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:08:51.159568 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:08:51.159568 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:08:51.159568 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 09:08:51.159568 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 09:08:51.159568 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 09:08:51.159568 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 09:08:51.160573 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:08:51.160573 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:08:51.160573 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 09:08:51.160573 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:08:51.160573 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 09:08:51.160573 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:08:51.160573 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:08:51.160573 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:08:51.160573 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:08:51.160573 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:08:51.160573 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:08:51.160573 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 24 09:08:51.160573 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:08:51.160573 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:08:51.160573 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:08:51.160573 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:08:51.160573 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:08:51.160573 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:08:51.161519 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:08:51.161519 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:08:51.161519 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 09:08:51.161519 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 09:08:51.161519 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 09:08:51.161519 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 09:08:51.161519 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 09:08:51.161519 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 09:08:51.161519 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 09:08:51.161519 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 24 09:08:51.161519 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 414/819] 10.222.0.16 () {56 vars in 1117 bytes} [Wed Aug 24 09:08:50 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:08:51.161519 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8123040-3f2f-47b0-833f-8bd1d3d1bdda None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:51.199462 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873241bd-4a2e-4142-8355-a384d64f5e73 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:51.238134 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839d3af6-2929-4751-8fe4-e47612e90a03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:51.238745 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839d3af6-2929-4751-8fe4-e47612e90a03 None admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:51.239024 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839d3af6-2929-4751-8fe4-e47612e90a03 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:51.239296 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839d3af6-2929-4751-8fe4-e47612e90a03 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:51.240134 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839d3af6-2929-4751-8fe4-e47612e90a03 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:51.241626 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839d3af6-2929-4751-8fe4-e47612e90a03 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:51.262788 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8123040-3f2f-47b0-833f-8bd1d3d1bdda None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:51.296312 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd46a35b-afa0-4dde-b006-1292e4e52016 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:51.296964 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd46a35b-afa0-4dde-b006-1292e4e52016 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:51.297244 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd46a35b-afa0-4dde-b006-1292e4e52016 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:51.297514 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd46a35b-afa0-4dde-b006-1292e4e52016 None admin] PATH_INFO: `/v3/services/network` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:51.298102 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd46a35b-afa0-4dde-b006-1292e4e52016 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:51.299463 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd46a35b-afa0-4dde-b006-1292e4e52016 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:51.309013 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-cd46a35b-afa0-4dde-b006-1292e4e52016 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 09:08:51.309013 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:08:51.309013 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 09:08:51.309013 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 09:08:51.309013 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 09:08:51.309013 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 09:08:51.309013 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 09:08:51.309013 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:08:51.309013 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 09:08:51.309013 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 24 09:08:51.309013 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 09:08:51.309013 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 24 09:08:51.309013 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:08:51.309013 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:08:51.309013 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:08:51.309013 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:08:51.309013 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:08:51.310199 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:08:51.310199 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:08:51.310199 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:08:51.310199 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:08:51.310199 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:08:51.310199 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:08:51.310199 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 09:08:51.310199 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 09:08:51.310199 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 09:08:51.310199 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 09:08:51.310199 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:08:51.310199 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:08:51.310199 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:08:51.310199 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:08:51.310199 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:08:51.310199 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:08:51.311126 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:08:51.311126 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 24 09:08:51.311126 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:08:51.311126 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:08:51.311126 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:08:51.311126 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:08:51.311126 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:08:51.311126 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:08:51.311126 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:08:51.311126 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:08:51.311126 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 09:08:51.311126 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:08:51.311126 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 09:08:51.311126 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 24 09:08:51.311126 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 415/820] 10.222.0.16 () {56 vars in 1111 bytes} [Wed Aug 24 09:08:51 2022] GET /identity/v3/services/network => generated 88 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:08:51.312046 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd46a35b-afa0-4dde-b006-1292e4e52016 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:51.348069 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 406/821] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 24 09:08:51 2022] POST /identity/v3/endpoints => generated 346 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:08:51.349130 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839d3af6-2929-4751-8fe4-e47612e90a03 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:51.387281 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:51.438001 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd46a35b-afa0-4dde-b006-1292e4e52016 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:51.483842 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e125f4d4-5b7b-4830-bd4c-a89d1b716275 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:51.484685 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e125f4d4-5b7b-4830-bd4c-a89d1b716275 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:51.485085 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e125f4d4-5b7b-4830-bd4c-a89d1b716275 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:51.485459 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e125f4d4-5b7b-4830-bd4c-a89d1b716275 None admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:51.486264 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e125f4d4-5b7b-4830-bd4c-a89d1b716275 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:51.488233 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e125f4d4-5b7b-4830-bd4c-a89d1b716275 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:51.488549 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e125f4d4-5b7b-4830-bd4c-a89d1b716275 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:51.496573 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839d3af6-2929-4751-8fe4-e47612e90a03 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:51.498771 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:51.509715 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 416/822] 10.222.0.16 () {56 vars in 1148 bytes} [Wed Aug 24 09:08:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:51.509715 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e125f4d4-5b7b-4830-bd4c-a89d1b716275 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:51.545791 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d825f8-d82f-42ad-84ed-35696409ac6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:51.546477 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d825f8-d82f-42ad-84ed-35696409ac6f None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:51.547504 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d825f8-d82f-42ad-84ed-35696409ac6f None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:51.547504 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d825f8-d82f-42ad-84ed-35696409ac6f None admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:51.547911 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d825f8-d82f-42ad-84ed-35696409ac6f None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:51.548219 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d825f8-d82f-42ad-84ed-35696409ac6f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:51.549666 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d825f8-d82f-42ad-84ed-35696409ac6f None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:51.576185 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 407/823] 10.222.0.16 () {56 vars in 1149 bytes} [Wed Aug 24 09:08:51 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:51.576185 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d825f8-d82f-42ad-84ed-35696409ac6f None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:51.645270 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e125f4d4-5b7b-4830-bd4c-a89d1b716275 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:51.668669 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:51.686276 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d825f8-d82f-42ad-84ed-35696409ac6f None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:51.691522 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdace6e6-beb2-416f-bba2-28b7b3004010 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:51.692201 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdace6e6-beb2-416f-bba2-28b7b3004010 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:51.692484 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdace6e6-beb2-416f-bba2-28b7b3004010 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:51.692757 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdace6e6-beb2-416f-bba2-28b7b3004010 None admin] PATH_INFO: `/v3/services` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:51.693394 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdace6e6-beb2-416f-bba2-28b7b3004010 None admin] RBAC: Adding query filter params (name=network) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:51.693697 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdace6e6-beb2-416f-bba2-28b7b3004010 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:51.695074 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdace6e6-beb2-416f-bba2-28b7b3004010 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:51.707000 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 417/824] 10.222.0.16 () {56 vars in 1112 bytes} [Wed Aug 24 09:08:51 2022] GET /identity/v3/services?name=network => generated 124 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:51.707000 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdace6e6-beb2-416f-bba2-28b7b3004010 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:51.735720 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37893502-005e-4b6c-a5d4-c1f5bfda27ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:51.736770 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37893502-005e-4b6c-a5d4-c1f5bfda27ae None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:51.737328 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37893502-005e-4b6c-a5d4-c1f5bfda27ae None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:51.737871 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37893502-005e-4b6c-a5d4-c1f5bfda27ae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:51.738976 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37893502-005e-4b6c-a5d4-c1f5bfda27ae None admin] RBAC: Adding query filter params (role.id=15f634d4a2d247ddab90c458e5e1c141, scope.project.id=c467e88e2fb0491bacefcf6d2ce326d9, user.id=1f055f7dc51d43bda9b315ca67f11d58) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:51.739609 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37893502-005e-4b6c-a5d4-c1f5bfda27ae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:51.741357 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37893502-005e-4b6c-a5d4-c1f5bfda27ae None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:51.748441 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 408/825] 10.222.0.16 () {56 vars in 1374 bytes} [Wed Aug 24 09:08:51 2022] GET /identity/v3/role_assignments?user.id=1f055f7dc51d43bda9b315ca67f11d58&scope.project.id=c467e88e2fb0491bacefcf6d2ce326d9&role.id=15f634d4a2d247ddab90c458e5e1c141 => generated 608 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:51.749529 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37893502-005e-4b6c-a5d4-c1f5bfda27ae None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:51.822539 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:51.905347 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37893502-005e-4b6c-a5d4-c1f5bfda27ae None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:51.915383 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdace6e6-beb2-416f-bba2-28b7b3004010 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:51.946885 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f84d8fc-3b79-46bb-b5fb-9d71b35d79ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:51.947624 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f84d8fc-3b79-46bb-b5fb-9d71b35d79ca None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:51.947905 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f84d8fc-3b79-46bb-b5fb-9d71b35d79ca None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:51.948227 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f84d8fc-3b79-46bb-b5fb-9d71b35d79ca None admin] PATH_INFO: `/v3/services` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:51.948985 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f84d8fc-3b79-46bb-b5fb-9d71b35d79ca None admin] RBAC: Adding query filter params (type=network) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:51.949413 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f84d8fc-3b79-46bb-b5fb-9d71b35d79ca None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:51.950790 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f84d8fc-3b79-46bb-b5fb-9d71b35d79ca None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:51.954924 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d50d0e-97ea-4ac3-98b6-43c5e1da419b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:51.955554 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d50d0e-97ea-4ac3-98b6-43c5e1da419b None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:51.955859 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d50d0e-97ea-4ac3-98b6-43c5e1da419b None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:51.956126 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d50d0e-97ea-4ac3-98b6-43c5e1da419b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:51.956322 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 409/826] 10.222.0.16 () {56 vars in 1112 bytes} [Wed Aug 24 09:08:51 2022] GET /identity/v3/services?type=network => generated 350 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:51.962533 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f84d8fc-3b79-46bb-b5fb-9d71b35d79ca None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:51.969513 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d50d0e-97ea-4ac3-98b6-43c5e1da419b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:51.970965 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d50d0e-97ea-4ac3-98b6-43c5e1da419b None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:51.991071 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-20d50d0e-97ea-4ac3-98b6-43c5e1da419b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:08:51.991071 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:08:51.991071 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:08:51.991071 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:08:51.991071 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:08:51.991071 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:08:51.991071 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:08:51.991071 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:08:51.991071 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:08:51.991071 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:08:51.991071 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:08:51.991071 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:08:51.991071 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:08:51.991071 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:08:51.991071 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:08:51.991071 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:08:51.992125 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:08:51.992125 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:08:51.992125 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:08:51.992125 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:08:51.992125 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:08:51.992125 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:08:51.992125 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:08:51.992125 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 24 09:08:51.992125 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:08:51.992125 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:08:51.992125 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:08:51.992125 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:08:51.992125 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:08:51.992125 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:08:51.992125 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:08:51.992125 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:08:51.992125 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:08:51.993019 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:08:51.993019 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:08:51.993019 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 24 09:08:51.993019 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 418/827] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 24 09:08:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:08:51.998812 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d50d0e-97ea-4ac3-98b6-43c5e1da419b None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:52.043390 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:52.084152 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:52.097918 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f84d8fc-3b79-46bb-b5fb-9d71b35d79ca None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:52.123931 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d50d0e-97ea-4ac3-98b6-43c5e1da419b None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:52.134200 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7954705-3ece-456b-bee0-fe43edcd049b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:52.134997 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7954705-3ece-456b-bee0-fe43edcd049b None admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:52.135363 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7954705-3ece-456b-bee0-fe43edcd049b None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:52.135796 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7954705-3ece-456b-bee0-fe43edcd049b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:52.136563 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7954705-3ece-456b-bee0-fe43edcd049b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:52.138036 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7954705-3ece-456b-bee0-fe43edcd049b None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:52.169095 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799e665d-26d8-4f15-8ec7-4539faed06a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:52.169979 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799e665d-26d8-4f15-8ec7-4539faed06a4 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:52.170403 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799e665d-26d8-4f15-8ec7-4539faed06a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:52.170878 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799e665d-26d8-4f15-8ec7-4539faed06a4 None admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:52.171749 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799e665d-26d8-4f15-8ec7-4539faed06a4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:52.172192 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799e665d-26d8-4f15-8ec7-4539faed06a4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:52.173841 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799e665d-26d8-4f15-8ec7-4539faed06a4 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:52.187289 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 419/828] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Aug 24 09:08:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:52.192309 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799e665d-26d8-4f15-8ec7-4539faed06a4 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:52.225021 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 410/829] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 24 09:08:51 2022] POST /identity/v3/endpoints => generated 337 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:08:52.316443 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:52.326248 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799e665d-26d8-4f15-8ec7-4539faed06a4 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:52.371611 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec5be15-06f7-43b6-b5df-31ad2c399c5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:52.372267 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec5be15-06f7-43b6-b5df-31ad2c399c5c None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:52.372546 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec5be15-06f7-43b6-b5df-31ad2c399c5c None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:52.372832 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec5be15-06f7-43b6-b5df-31ad2c399c5c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:52.380210 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec5be15-06f7-43b6-b5df-31ad2c399c5c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:52.381653 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec5be15-06f7-43b6-b5df-31ad2c399c5c None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:52.393798 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-bec5be15-06f7-43b6-b5df-31ad2c399c5c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:08:52.393798 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:08:52.393798 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:08:52.393798 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:08:52.393798 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:08:52.393798 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:08:52.393798 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:08:52.393798 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:08:52.393798 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:08:52.393798 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:08:52.393798 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:08:52.393798 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:08:52.393798 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 09:08:52.393798 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 09:08:52.393798 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 09:08:52.394845 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 09:08:52.394845 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:08:52.394845 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:08:52.394845 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:08:52.394845 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:08:52.394845 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:08:52.394845 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:08:52.394845 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:08:52.394845 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 24 09:08:52.394845 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:08:52.394845 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:08:52.394845 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:08:52.394845 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:08:52.394845 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:08:52.394845 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:08:52.394845 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:08:52.394845 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:08:52.396200 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 09:08:52.396200 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 09:08:52.396200 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 09:08:52.396200 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 09:08:52.396200 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 09:08:52.396200 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 09:08:52.396200 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:08:52.396200 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 24 09:08:52.396200 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 420/830] 10.222.0.16 () {56 vars in 1111 bytes} [Wed Aug 24 09:08:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:08:52.404742 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7954705-3ece-456b-bee0-fe43edcd049b None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:52.544736 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7954705-3ece-456b-bee0-fe43edcd049b None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:52.589723 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b92509-4984-4bc6-929c-0cc91abd3326 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:52.590353 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b92509-4984-4bc6-929c-0cc91abd3326 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:52.590636 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b92509-4984-4bc6-929c-0cc91abd3326 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:52.590912 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b92509-4984-4bc6-929c-0cc91abd3326 None admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:52.591717 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b92509-4984-4bc6-929c-0cc91abd3326 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:52.592030 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b92509-4984-4bc6-929c-0cc91abd3326 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:52.593556 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b92509-4984-4bc6-929c-0cc91abd3326 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:52.608886 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 411/831] 10.222.0.16 () {56 vars in 1148 bytes} [Wed Aug 24 09:08:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:52.616746 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec5be15-06f7-43b6-b5df-31ad2c399c5c None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:52.726817 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec5be15-06f7-43b6-b5df-31ad2c399c5c None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:52.749992 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ab81b3-e2b5-4291-a7cc-7bbbe4d28f00 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:52.750320 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ab81b3-e2b5-4291-a7cc-7bbbe4d28f00 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:52.750551 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ab81b3-e2b5-4291-a7cc-7bbbe4d28f00 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:52.752694 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 412/832] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:08:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 09:08:52.757963 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247d67bc-fb7f-41b6-97f2-a302d4d0e9b8 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:52.758535 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247d67bc-fb7f-41b6-97f2-a302d4d0e9b8 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:52.758535 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247d67bc-fb7f-41b6-97f2-a302d4d0e9b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:52.772011 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67d47d1-7fc5-4cd5-b6ca-172223bed4cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:52.773134 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67d47d1-7fc5-4cd5-b6ca-172223bed4cc None admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:52.773640 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67d47d1-7fc5-4cd5-b6ca-172223bed4cc None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:52.774129 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67d47d1-7fc5-4cd5-b6ca-172223bed4cc None admin] PATH_INFO: `/v3/projects/c467e88e2fb0491bacefcf6d2ce326d9/users/0489462287fc4aea8e8e720af24e1ef8/roles/67fd70a7d9794f7fb8faebffdec46dd2` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:52.775990 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:52.778605 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-247d67bc-fb7f-41b6-97f2-a302d4d0e9b8 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:08:52.779886 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-247d67bc-fb7f-41b6-97f2-a302d4d0e9b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:52.790440 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67d47d1-7fc5-4cd5-b6ca-172223bed4cc None admin] RBAC: Authorizing `identity:create_grant(project_id=c467e88e2fb0491bacefcf6d2ce326d9, user_id=0489462287fc4aea8e8e720af24e1ef8, role_id=67fd70a7d9794f7fb8faebffdec46dd2)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:52.799948 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67d47d1-7fc5-4cd5-b6ca-172223bed4cc None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:52.852844 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 413/833] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:08:52 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 09:08:52.865938 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:52.871694 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 421/834] 10.222.0.16 () {58 vars in 1439 bytes} [Wed Aug 24 09:08:52 2022] PUT /identity/v3/projects/c467e88e2fb0491bacefcf6d2ce326d9/users/0489462287fc4aea8e8e720af24e1ef8/roles/67fd70a7d9794f7fb8faebffdec46dd2 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:08:53.122093 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247d67bc-fb7f-41b6-97f2-a302d4d0e9b8 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:53.232271 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247d67bc-fb7f-41b6-97f2-a302d4d0e9b8 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:53.396494 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ff4b4e-2a5a-433d-a813-f9b75b524b38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:53.397497 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ff4b4e-2a5a-433d-a813-f9b75b524b38 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:53.397991 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ff4b4e-2a5a-433d-a813-f9b75b524b38 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:53.398475 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ff4b4e-2a5a-433d-a813-f9b75b524b38 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:53.401256 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:53.410901 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ff4b4e-2a5a-433d-a813-f9b75b524b38 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:08:53.416836 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-88ff4b4e-2a5a-433d-a813-f9b75b524b38 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:08:53.416836 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:08:53.416836 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:08:53.416836 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:08:53.416836 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:08:53.416836 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:08:53.416836 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:08:53.416836 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:08:53.416836 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:08:53.416836 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:08:53.416836 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:08:53.416836 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:08:53.416836 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:08:53.416836 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:08:53.416836 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:08:53.416836 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:08:53.424171 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:08:53.424171 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:08:53.424171 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:08:53.424171 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:08:53.424171 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:08:53.424171 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:08:53.424171 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:08:53.424171 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:08:53.424171 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:08:53.424171 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:08:53.424171 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:08:53.424171 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Aug 24 09:08:53.424171 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:08:53.424171 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:08:53.424171 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:08:53.424171 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:08:53.425275 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:08:53.425275 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:08:53.425275 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:08:53.425275 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:08:53.425275 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:08:53.425275 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:08:53.425275 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:08:53.425275 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:08:53.425275 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:08:53.425275 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:08:53.425275 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:08:53.425275 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Aug 24 09:08:53.425275 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 414/835] 10.222.0.16 () {56 vars in 1096 bytes} [Wed Aug 24 09:08:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:08:53.425990 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67d47d1-7fc5-4cd5-b6ca-172223bed4cc None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:53.492287 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bae9008-4f07-4b4b-aee4-0dc06846e399 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:53.492639 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bae9008-4f07-4b4b-aee4-0dc06846e399 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:53.493034 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bae9008-4f07-4b4b-aee4-0dc06846e399 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:53.495871 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 415/836] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:08:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 09:08:53.500757 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba2d161-9aa6-4c56-8f2e-1e5edc3006e5 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:53.501034 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba2d161-9aa6-4c56-8f2e-1e5edc3006e5 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:53.501302 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba2d161-9aa6-4c56-8f2e-1e5edc3006e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:53.519129 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-aba2d161-9aa6-4c56-8f2e-1e5edc3006e5 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:08:53.520769 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-aba2d161-9aa6-4c56-8f2e-1e5edc3006e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:53.535698 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67d47d1-7fc5-4cd5-b6ca-172223bed4cc None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:53.573018 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca20571e-2b97-4e31-8cb8-c23ab612a4b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:53.574290 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca20571e-2b97-4e31-8cb8-c23ab612a4b1 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:53.574785 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca20571e-2b97-4e31-8cb8-c23ab612a4b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:53.575261 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca20571e-2b97-4e31-8cb8-c23ab612a4b1 None admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:53.576310 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca20571e-2b97-4e31-8cb8-c23ab612a4b1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:53.576856 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca20571e-2b97-4e31-8cb8-c23ab612a4b1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:53.578634 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca20571e-2b97-4e31-8cb8-c23ab612a4b1 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:53.593444 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 422/837] 10.222.0.16 () {56 vars in 1099 bytes} [Wed Aug 24 09:08:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:53.594699 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca20571e-2b97-4e31-8cb8-c23ab612a4b1 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:53.608072 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 416/838] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:08:53 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 09:08:53.660932 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:53.709378 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca20571e-2b97-4e31-8cb8-c23ab612a4b1 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:53.746273 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2cc552-a891-4117-a0d5-ec1293723a7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:53.747692 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2cc552-a891-4117-a0d5-ec1293723a7f None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:53.748166 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2cc552-a891-4117-a0d5-ec1293723a7f None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:53.748631 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2cc552-a891-4117-a0d5-ec1293723a7f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:53.760046 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2cc552-a891-4117-a0d5-ec1293723a7f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:53.761700 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2cc552-a891-4117-a0d5-ec1293723a7f None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:53.778438 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-fa2cc552-a891-4117-a0d5-ec1293723a7f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:08:53.778438 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:08:53.778438 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:08:53.778438 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:08:53.778438 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:08:53.778438 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:08:53.778438 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:08:53.778438 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:08:53.778438 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:08:53.778438 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:08:53.778438 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:08:53.778438 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:08:53.778438 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:08:53.778438 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:08:53.778438 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:08:53.778438 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:08:53.779443 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:08:53.779443 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:08:53.779443 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:08:53.779443 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:08:53.779443 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:08:53.779443 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:08:53.779443 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:08:53.779443 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 24 09:08:53.779443 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:08:53.779443 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:08:53.779443 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:08:53.779443 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:08:53.779443 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:08:53.779443 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:08:53.779443 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:08:53.779443 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:08:53.779443 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:08:53.780377 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:08:53.780377 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:08:53.780377 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 24 09:08:53.782186 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 423/839] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 24 09:08:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:08:53.788836 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba2d161-9aa6-4c56-8f2e-1e5edc3006e5 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:53.896169 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba2d161-9aa6-4c56-8f2e-1e5edc3006e5 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:53.900739 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2cc552-a891-4117-a0d5-ec1293723a7f None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:53.922041 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:53.934698 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466a4b94-cd37-4290-8605-9383168df943 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:53.935322 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466a4b94-cd37-4290-8605-9383168df943 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:53.935672 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466a4b94-cd37-4290-8605-9383168df943 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:53.935890 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466a4b94-cd37-4290-8605-9383168df943 None admin] PATH_INFO: `/v3/domains` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:53.936618 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466a4b94-cd37-4290-8605-9383168df943 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:53.936926 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466a4b94-cd37-4290-8605-9383168df943 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:53.938435 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466a4b94-cd37-4290-8605-9383168df943 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:53.952633 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 417/840] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Aug 24 09:08:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:53.958610 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466a4b94-cd37-4290-8605-9383168df943 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:54.020994 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2cc552-a891-4117-a0d5-ec1293723a7f None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:54.045695 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:54.071298 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e98af5-6ec0-4dea-874b-4d29a8d737b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:54.072058 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e98af5-6ec0-4dea-874b-4d29a8d737b7 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:54.072367 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e98af5-6ec0-4dea-874b-4d29a8d737b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:54.072636 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e98af5-6ec0-4dea-874b-4d29a8d737b7 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:54.073472 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e98af5-6ec0-4dea-874b-4d29a8d737b7 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:54.074818 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e98af5-6ec0-4dea-874b-4d29a8d737b7 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:54.087984 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-08e98af5-6ec0-4dea-874b-4d29a8d737b7 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 09:08:54.087984 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:08:54.087984 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 09:08:54.087984 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 09:08:54.087984 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 09:08:54.087984 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 09:08:54.087984 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 09:08:54.087984 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:08:54.087984 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 09:08:54.087984 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 24 09:08:54.087984 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 09:08:54.087984 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 24 09:08:54.087984 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:08:54.087984 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:08:54.087984 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:08:54.087984 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:08:54.087984 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:08:54.089015 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:08:54.089015 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:08:54.089015 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:08:54.089015 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:08:54.089015 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:08:54.089015 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:08:54.089015 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 09:08:54.089015 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 09:08:54.089015 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 09:08:54.089015 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 09:08:54.089015 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:08:54.089015 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:08:54.089015 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:08:54.089015 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:08:54.089015 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:08:54.089015 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:08:54.089873 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:08:54.089873 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 24 09:08:54.089873 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:08:54.089873 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:08:54.089873 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:08:54.089873 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:08:54.089873 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:08:54.089873 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:08:54.089873 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:08:54.089873 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:08:54.089873 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 09:08:54.089873 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:08:54.089873 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 09:08:54.089873 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 24 09:08:54.093906 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 424/841] 10.222.0.16 () {56 vars in 1114 bytes} [Wed Aug 24 09:08:53 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:08:54.095145 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466a4b94-cd37-4290-8605-9383168df943 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:54.095263 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e98af5-6ec0-4dea-874b-4d29a8d737b7 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:54.140713 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6872ec-fc1f-4517-aa3c-674fd74bbbca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:54.142173 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6872ec-fc1f-4517-aa3c-674fd74bbbca None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:54.142675 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6872ec-fc1f-4517-aa3c-674fd74bbbca None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:54.143563 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6872ec-fc1f-4517-aa3c-674fd74bbbca None admin] PATH_INFO: `/v3/users/swift` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:54.154432 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6872ec-fc1f-4517-aa3c-674fd74bbbca None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:54.156750 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6872ec-fc1f-4517-aa3c-674fd74bbbca None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:54.174051 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-0e6872ec-fc1f-4517-aa3c-674fd74bbbca None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 09:08:54.174051 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:08:54.174051 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:08:54.174051 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:08:54.174051 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:08:54.174051 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:08:54.174051 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:08:54.174051 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:08:54.174051 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:08:54.174051 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:08:54.174051 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:08:54.174051 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:08:54.174051 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 09:08:54.174051 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 09:08:54.174051 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 09:08:54.174051 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 09:08:54.175008 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:08:54.175008 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:08:54.175008 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 09:08:54.175008 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:08:54.175008 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 09:08:54.175008 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:08:54.175008 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:08:54.175008 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:08:54.175008 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:08:54.175008 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:08:54.175008 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:08:54.175008 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Aug 24 09:08:54.175008 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:08:54.175008 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:08:54.175008 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:08:54.175008 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:08:54.175008 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:08:54.175008 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:08:54.176033 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:08:54.176033 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:08:54.176033 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 09:08:54.176033 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 09:08:54.176033 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 09:08:54.176033 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 09:08:54.176033 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 09:08:54.176033 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 09:08:54.176033 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 09:08:54.176033 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Aug 24 09:08:54.184257 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 418/842] 10.222.0.16 () {56 vars in 1096 bytes} [Wed Aug 24 09:08:53 2022] GET /identity/v3/users/swift => generated 83 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:08:54.185061 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cf4b5a-d167-4a87-ad50-c523cfa73b48 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:54.185385 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cf4b5a-d167-4a87-ad50-c523cfa73b48 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:54.185659 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cf4b5a-d167-4a87-ad50-c523cfa73b48 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:54.187122 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 419/843] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:08:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 09:08:54.187764 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cf4b5a-d167-4a87-ad50-c523cfa73b48 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:54.215878 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e98af5-6ec0-4dea-874b-4d29a8d737b7 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:54.239027 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:54.254634 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3236be8a-3120-4d1c-851e-682328e00c04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:54.255397 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3236be8a-3120-4d1c-851e-682328e00c04 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:54.255987 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3236be8a-3120-4d1c-851e-682328e00c04 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:54.256276 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3236be8a-3120-4d1c-851e-682328e00c04 None admin] PATH_INFO: `/v3/services` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:54.257055 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3236be8a-3120-4d1c-851e-682328e00c04 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:54.257358 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3236be8a-3120-4d1c-851e-682328e00c04 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:54.258712 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3236be8a-3120-4d1c-851e-682328e00c04 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:54.263479 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 425/844] 10.222.0.16 () {56 vars in 1114 bytes} [Wed Aug 24 09:08:54 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:54.269263 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd10e28-ec26-45d1-ae33-c0bcbc0e5e26 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:54.269550 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd10e28-ec26-45d1-ae33-c0bcbc0e5e26 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:54.269820 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd10e28-ec26-45d1-ae33-c0bcbc0e5e26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:54.288502 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-ebd10e28-ec26-45d1-ae33-c0bcbc0e5e26 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:08:54.289728 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-ebd10e28-ec26-45d1-ae33-c0bcbc0e5e26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:54.301069 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cf4b5a-d167-4a87-ad50-c523cfa73b48 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:54.332551 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:54.336950 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c986567d-7f12-4f0a-bcaa-475b36909621 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:54.338092 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c986567d-7f12-4f0a-bcaa-475b36909621 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:54.338383 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c986567d-7f12-4f0a-bcaa-475b36909621 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:54.338671 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c986567d-7f12-4f0a-bcaa-475b36909621 None admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:54.339396 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c986567d-7f12-4f0a-bcaa-475b36909621 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:54.339735 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c986567d-7f12-4f0a-bcaa-475b36909621 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:54.341168 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c986567d-7f12-4f0a-bcaa-475b36909621 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:54.360031 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 420/845] 10.222.0.16 () {56 vars in 1135 bytes} [Wed Aug 24 09:08:54 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:54.361160 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c986567d-7f12-4f0a-bcaa-475b36909621 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:54.375646 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 426/846] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:08:54 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 09:08:54.375646 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd10e28-ec26-45d1-ae33-c0bcbc0e5e26 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:54.469972 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c986567d-7f12-4f0a-bcaa-475b36909621 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:54.471738 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:54.502577 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd10e28-ec26-45d1-ae33-c0bcbc0e5e26 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:54.507518 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d1589d-62d7-417e-842b-edf91e4c1295 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:54.508157 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d1589d-62d7-417e-842b-edf91e4c1295 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:54.508433 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d1589d-62d7-417e-842b-edf91e4c1295 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:54.508704 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d1589d-62d7-417e-842b-edf91e4c1295 None admin] PATH_INFO: `/v3/services` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:54.509332 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d1589d-62d7-417e-842b-edf91e4c1295 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:54.509860 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d1589d-62d7-417e-842b-edf91e4c1295 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:54.511237 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d1589d-62d7-417e-842b-edf91e4c1295 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:54.524416 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 421/847] 10.222.0.16 () {56 vars in 1114 bytes} [Wed Aug 24 09:08:54 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:54.543341 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98bfc5c-3476-4d38-9302-af277c112387 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:54.544081 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98bfc5c-3476-4d38-9302-af277c112387 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:54.544359 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98bfc5c-3476-4d38-9302-af277c112387 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:54.544631 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98bfc5c-3476-4d38-9302-af277c112387 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:54.590278 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:54.621729 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98bfc5c-3476-4d38-9302-af277c112387 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:54.623196 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98bfc5c-3476-4d38-9302-af277c112387 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:54.645902 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d1589d-62d7-417e-842b-edf91e4c1295 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:54.748138 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 427/848] 10.222.0.16 () {58 vars in 1282 bytes} [Wed Aug 24 09:08:54 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 09:08:54.753354 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98bfc5c-3476-4d38-9302-af277c112387 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:54.790521 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d1589d-62d7-417e-842b-edf91e4c1295 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:54.830012 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:54.832069 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a579c0e2-d400-4b6e-90ec-dfb9fd83b85a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:54.832772 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a579c0e2-d400-4b6e-90ec-dfb9fd83b85a None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:54.833119 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a579c0e2-d400-4b6e-90ec-dfb9fd83b85a None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:54.833462 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a579c0e2-d400-4b6e-90ec-dfb9fd83b85a None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:54.834114 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a579c0e2-d400-4b6e-90ec-dfb9fd83b85a None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:54.835552 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a579c0e2-d400-4b6e-90ec-dfb9fd83b85a None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:54.845453 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-a579c0e2-d400-4b6e-90ec-dfb9fd83b85a None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 09:08:54.845453 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:08:54.845453 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 09:08:54.845453 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 09:08:54.845453 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 09:08:54.845453 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 09:08:54.845453 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 09:08:54.845453 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:08:54.845453 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 09:08:54.845453 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Aug 24 09:08:54.845453 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 09:08:54.845453 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Aug 24 09:08:54.845453 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:08:54.845453 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:08:54.845453 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:08:54.845453 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:08:54.846415 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:08:54.846415 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:08:54.846415 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:08:54.846415 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:08:54.846415 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:08:54.846415 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:08:54.846415 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:08:54.846415 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 09:08:54.846415 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 09:08:54.846415 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 09:08:54.846415 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 09:08:54.846415 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:08:54.846415 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:08:54.846415 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:08:54.846415 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:08:54.846415 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:08:54.846415 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:08:54.847327 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:08:54.847327 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Aug 24 09:08:54.847327 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:08:54.847327 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:08:54.847327 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:08:54.847327 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:08:54.847327 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:08:54.847327 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:08:54.847327 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:08:54.847327 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:08:54.847327 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 09:08:54.847327 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:08:54.847327 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 09:08:54.847327 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Aug 24 09:08:54.847327 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 422/849] 10.222.0.16 () {56 vars in 1132 bytes} [Wed Aug 24 09:08:54 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:08:54.848769 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cb6baa-0aef-47c5-b93b-4c6030d25a87 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:54.848769 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cb6baa-0aef-47c5-b93b-4c6030d25a87 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:54.848769 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cb6baa-0aef-47c5-b93b-4c6030d25a87 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:54.850316 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 423/850] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:08:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:08:54.852467 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cb6baa-0aef-47c5-b93b-4c6030d25a87 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:54.865784 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98bfc5c-3476-4d38-9302-af277c112387 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:54.886501 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:54.902948 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fc7d06-7a6f-4c44-8fd4-6c6c1d43ad04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:54.903605 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fc7d06-7a6f-4c44-8fd4-6c6c1d43ad04 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:54.903888 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fc7d06-7a6f-4c44-8fd4-6c6c1d43ad04 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:54.904163 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fc7d06-7a6f-4c44-8fd4-6c6c1d43ad04 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:54.914164 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fc7d06-7a6f-4c44-8fd4-6c6c1d43ad04 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:54.915650 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fc7d06-7a6f-4c44-8fd4-6c6c1d43ad04 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:54.930625 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-f7fc7d06-7a6f-4c44-8fd4-6c6c1d43ad04 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:08:54.930625 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:08:54.930625 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:08:54.930625 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:08:54.930625 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:08:54.930625 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:08:54.930625 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:08:54.930625 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:08:54.930625 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:08:54.930625 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:08:54.930625 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:08:54.930625 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:08:54.930625 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:08:54.930625 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:08:54.930625 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:08:54.930625 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:08:54.937423 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:08:54.937423 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:08:54.937423 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:08:54.937423 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:08:54.937423 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:08:54.937423 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:08:54.937423 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:08:54.937423 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 24 09:08:54.937423 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:08:54.937423 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:08:54.937423 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:08:54.937423 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:08:54.937423 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:08:54.937423 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:08:54.937423 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:08:54.937423 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:08:54.937423 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:08:54.938503 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:08:54.938503 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:08:54.938503 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 24 09:08:54.938503 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 428/851] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 24 09:08:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:08:54.938503 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fbd648-28b4-4860-be7a-fe96c31404d8 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:54.938503 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fbd648-28b4-4860-be7a-fe96c31404d8 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:54.938503 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fbd648-28b4-4860-be7a-fe96c31404d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:54.952691 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-d8fbd648-28b4-4860-be7a-fe96c31404d8 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:08:54.954328 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-d8fbd648-28b4-4860-be7a-fe96c31404d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:54.972981 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cb6baa-0aef-47c5-b93b-4c6030d25a87 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:55.012133 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82512c0-442c-4ae8-9616-6f04171dc5a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:55.013186 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82512c0-442c-4ae8-9616-6f04171dc5a9 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:55.013513 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82512c0-442c-4ae8-9616-6f04171dc5a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:55.013801 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82512c0-442c-4ae8-9616-6f04171dc5a9 None admin] PATH_INFO: `/v3/services` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:55.014406 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82512c0-442c-4ae8-9616-6f04171dc5a9 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:55.014709 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82512c0-442c-4ae8-9616-6f04171dc5a9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:55.016112 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82512c0-442c-4ae8-9616-6f04171dc5a9 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:55.021147 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 424/852] 10.222.0.16 () {56 vars in 1126 bytes} [Wed Aug 24 09:08:54 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:55.021923 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82512c0-442c-4ae8-9616-6f04171dc5a9 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:55.033183 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 429/853] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:08:54 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 09:08:55.034089 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fbd648-28b4-4860-be7a-fe96c31404d8 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:55.105230 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:55.136110 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:55.144496 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82512c0-442c-4ae8-9616-6f04171dc5a9 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:55.167837 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fbd648-28b4-4860-be7a-fe96c31404d8 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:55.183301 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9561c593-2fdc-4834-82b8-bebf46d0633e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:55.183787 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9561c593-2fdc-4834-82b8-bebf46d0633e None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:55.184101 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9561c593-2fdc-4834-82b8-bebf46d0633e None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:55.185372 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9561c593-2fdc-4834-82b8-bebf46d0633e None admin] PATH_INFO: `/v3/domains` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:55.185372 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9561c593-2fdc-4834-82b8-bebf46d0633e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:55.185543 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9561c593-2fdc-4834-82b8-bebf46d0633e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:55.186880 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9561c593-2fdc-4834-82b8-bebf46d0633e None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:55.204264 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 425/854] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Aug 24 09:08:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:55.206735 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9561c593-2fdc-4834-82b8-bebf46d0633e None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:55.212134 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc13c33-2882-4527-8c1b-beb79e302da0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:55.213536 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc13c33-2882-4527-8c1b-beb79e302da0 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:55.214015 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc13c33-2882-4527-8c1b-beb79e302da0 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:55.214501 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc13c33-2882-4527-8c1b-beb79e302da0 None admin] PATH_INFO: `/v3/services` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:55.215381 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc13c33-2882-4527-8c1b-beb79e302da0 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:55.216341 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc13c33-2882-4527-8c1b-beb79e302da0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:55.218366 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc13c33-2882-4527-8c1b-beb79e302da0 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:55.225067 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 430/855] 10.222.0.16 () {56 vars in 1126 bytes} [Wed Aug 24 09:08:55 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:55.308841 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc13c33-2882-4527-8c1b-beb79e302da0 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:55.347699 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9561c593-2fdc-4834-82b8-bebf46d0633e None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:55.401706 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117cb0dd-f40f-46f1-bb33-ffe02571ebd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:55.402345 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117cb0dd-f40f-46f1-bb33-ffe02571ebd8 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:55.402618 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117cb0dd-f40f-46f1-bb33-ffe02571ebd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:55.402885 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117cb0dd-f40f-46f1-bb33-ffe02571ebd8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:55.411380 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117cb0dd-f40f-46f1-bb33-ffe02571ebd8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:55.412856 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117cb0dd-f40f-46f1-bb33-ffe02571ebd8 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:55.425803 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-117cb0dd-f40f-46f1-bb33-ffe02571ebd8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:08:55.425803 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:08:55.425803 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:08:55.425803 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:08:55.425803 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:08:55.425803 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:08:55.425803 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:08:55.425803 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:08:55.425803 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:08:55.425803 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:08:55.425803 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:08:55.425803 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:08:55.425803 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 09:08:55.425803 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 09:08:55.425803 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 09:08:55.426706 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 09:08:55.426706 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:08:55.426706 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:08:55.426706 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:08:55.426706 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:08:55.426706 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:08:55.426706 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:08:55.426706 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:08:55.426706 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Aug 24 09:08:55.426706 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:08:55.426706 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:08:55.426706 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:08:55.426706 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:08:55.426706 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:08:55.426706 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:08:55.426706 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:08:55.426706 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:08:55.427632 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 09:08:55.427632 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 09:08:55.427632 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 09:08:55.427632 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 09:08:55.427632 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 09:08:55.427632 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 09:08:55.427632 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:08:55.427632 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Aug 24 09:08:55.427632 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 426/856] 10.222.0.16 () {56 vars in 1111 bytes} [Wed Aug 24 09:08:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 09:08:55.432257 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc13c33-2882-4527-8c1b-beb79e302da0 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:55.433471 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117cb0dd-f40f-46f1-bb33-ffe02571ebd8 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:55.452249 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:55.457108 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:55.480428 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25646c64-8ae5-4573-8798-fd93d018084e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:55.481732 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25646c64-8ae5-4573-8798-fd93d018084e None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:55.482489 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25646c64-8ae5-4573-8798-fd93d018084e None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:55.482961 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25646c64-8ae5-4573-8798-fd93d018084e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:55.496363 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25646c64-8ae5-4573-8798-fd93d018084e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:08:55.501922 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-25646c64-8ae5-4573-8798-fd93d018084e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 09:08:55.501922 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:08:55.501922 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:08:55.501922 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:08:55.501922 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:08:55.501922 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:08:55.501922 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:08:55.501922 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:08:55.501922 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:08:55.501922 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:08:55.501922 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:08:55.501922 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:08:55.501922 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:08:55.501922 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:08:55.501922 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:08:55.501922 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:08:55.503000 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:08:55.503000 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:08:55.503000 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:08:55.503000 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:08:55.503000 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:08:55.503000 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:08:55.503000 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:08:55.503000 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:08:55.503000 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:08:55.503000 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:08:55.503000 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:08:55.503000 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 24 09:08:55.503000 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:08:55.503000 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:08:55.503000 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:08:55.503000 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:08:55.503897 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:08:55.503897 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:08:55.503897 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:08:55.503897 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:08:55.503897 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:08:55.503897 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:08:55.503897 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:08:55.503897 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:08:55.503897 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:08:55.503897 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:08:55.503897 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 09:08:55.503897 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 24 09:08:55.506413 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 431/857] 10.222.0.16 () {56 vars in 1102 bytes} [Wed Aug 24 09:08:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 09:08:55.512688 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25646c64-8ae5-4573-8798-fd93d018084e None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:55.549895 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117cb0dd-f40f-46f1-bb33-ffe02571ebd8 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:55.586602 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6371b48a-ecff-4ed3-9cec-9b4b847e5cc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:55.587231 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6371b48a-ecff-4ed3-9cec-9b4b847e5cc0 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:55.587656 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6371b48a-ecff-4ed3-9cec-9b4b847e5cc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:55.588026 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6371b48a-ecff-4ed3-9cec-9b4b847e5cc0 None admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:55.588804 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6371b48a-ecff-4ed3-9cec-9b4b847e5cc0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:55.589180 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6371b48a-ecff-4ed3-9cec-9b4b847e5cc0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:55.590730 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6371b48a-ecff-4ed3-9cec-9b4b847e5cc0 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:55.608785 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 427/858] 10.222.0.16 () {56 vars in 1148 bytes} [Wed Aug 24 09:08:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:08:55.612269 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6371b48a-ecff-4ed3-9cec-9b4b847e5cc0 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:55.647310 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25646c64-8ae5-4573-8798-fd93d018084e None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:55.696477 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:55.701714 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c75a5b-3392-4786-a355-a0ee5bd38897 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:55.702779 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c75a5b-3392-4786-a355-a0ee5bd38897 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:55.703221 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c75a5b-3392-4786-a355-a0ee5bd38897 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:55.704096 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c75a5b-3392-4786-a355-a0ee5bd38897 None admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:55.713566 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c75a5b-3392-4786-a355-a0ee5bd38897 None admin] RBAC: Adding query filter params (name=service) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:55.713566 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c75a5b-3392-4786-a355-a0ee5bd38897 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:55.713566 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c75a5b-3392-4786-a355-a0ee5bd38897 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:55.722446 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 432/859] 10.222.0.16 () {56 vars in 1103 bytes} [Wed Aug 24 09:08:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:55.723504 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c75a5b-3392-4786-a355-a0ee5bd38897 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:55.729335 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6371b48a-ecff-4ed3-9cec-9b4b847e5cc0 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:55.771863 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf8fa76-3bb3-497d-a1b0-aca375019988 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:55.772630 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf8fa76-3bb3-497d-a1b0-aca375019988 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:55.773130 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf8fa76-3bb3-497d-a1b0-aca375019988 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:55.773501 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf8fa76-3bb3-497d-a1b0-aca375019988 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:55.774332 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf8fa76-3bb3-497d-a1b0-aca375019988 None admin] RBAC: Adding query filter params (role.id=15f634d4a2d247ddab90c458e5e1c141, scope.project.id=c467e88e2fb0491bacefcf6d2ce326d9, user.id=fdd5f0d67fe54e9fae6f4af644988d93) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:55.775721 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf8fa76-3bb3-497d-a1b0-aca375019988 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:55.776041 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:55.778998 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf8fa76-3bb3-497d-a1b0-aca375019988 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:55.785334 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 428/860] 10.222.0.16 () {56 vars in 1374 bytes} [Wed Aug 24 09:08:55 2022] GET /identity/v3/role_assignments?user.id=fdd5f0d67fe54e9fae6f4af644988d93&scope.project.id=c467e88e2fb0491bacefcf6d2ce326d9&role.id=15f634d4a2d247ddab90c458e5e1c141 => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:55.851882 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c75a5b-3392-4786-a355-a0ee5bd38897 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:55.890196 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8755beb-1b39-46c5-b34a-2f7f060ae44b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:55.891259 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8755beb-1b39-46c5-b34a-2f7f060ae44b None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:55.891791 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8755beb-1b39-46c5-b34a-2f7f060ae44b None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:55.892354 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8755beb-1b39-46c5-b34a-2f7f060ae44b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:55.905839 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8755beb-1b39-46c5-b34a-2f7f060ae44b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:55.907712 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8755beb-1b39-46c5-b34a-2f7f060ae44b None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:55.924603 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-e8755beb-1b39-46c5-b34a-2f7f060ae44b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:08:55.924603 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:08:55.924603 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:08:55.924603 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:08:55.924603 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:08:55.924603 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:08:55.924603 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:08:55.924603 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:08:55.924603 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:08:55.924603 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:08:55.924603 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:08:55.924603 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:08:55.924603 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:08:55.924603 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:08:55.924603 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:08:55.924603 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:08:55.925951 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:08:55.925951 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:08:55.925951 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:08:55.925951 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:08:55.925951 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:08:55.925951 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:08:55.925951 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:08:55.925951 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 24 09:08:55.925951 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:08:55.925951 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:08:55.925951 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:08:55.925951 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:08:55.925951 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:08:55.925951 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:08:55.925951 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:08:55.925951 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:08:55.925951 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:08:55.926859 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:08:55.926859 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:08:55.926859 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 24 09:08:55.929082 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 433/861] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 24 09:08:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:08:55.935178 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf8fa76-3bb3-497d-a1b0-aca375019988 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:56.044439 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf8fa76-3bb3-497d-a1b0-aca375019988 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:56.079735 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50699d0-5411-4715-aaf1-c6a37b5f2f92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:56.080693 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50699d0-5411-4715-aaf1-c6a37b5f2f92 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:56.081202 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50699d0-5411-4715-aaf1-c6a37b5f2f92 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:56.081675 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50699d0-5411-4715-aaf1-c6a37b5f2f92 None admin] PATH_INFO: `/v3/domains` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:56.082525 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50699d0-5411-4715-aaf1-c6a37b5f2f92 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:56.083027 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50699d0-5411-4715-aaf1-c6a37b5f2f92 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:56.084612 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50699d0-5411-4715-aaf1-c6a37b5f2f92 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:56.092395 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:56.098517 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 429/862] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Aug 24 09:08:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:56.104667 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8755beb-1b39-46c5-b34a-2f7f060ae44b None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:56.201035 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:56.211137 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8755beb-1b39-46c5-b34a-2f7f060ae44b None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:56.247458 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2993b7d6-9131-43ac-9e84-9646c19db515 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:56.248368 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2993b7d6-9131-43ac-9e84-9646c19db515 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:56.248754 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2993b7d6-9131-43ac-9e84-9646c19db515 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:56.249123 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2993b7d6-9131-43ac-9e84-9646c19db515 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:56.256855 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2993b7d6-9131-43ac-9e84-9646c19db515 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:56.258476 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2993b7d6-9131-43ac-9e84-9646c19db515 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:56.271699 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-2993b7d6-9131-43ac-9e84-9646c19db515 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 09:08:56.271699 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:08:56.271699 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:08:56.271699 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:08:56.271699 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:08:56.271699 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:08:56.271699 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:08:56.271699 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:08:56.271699 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:08:56.271699 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:08:56.271699 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:08:56.271699 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:08:56.271699 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 09:08:56.271699 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 09:08:56.271699 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 09:08:56.271699 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 09:08:56.272858 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:08:56.272858 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:08:56.272858 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 09:08:56.272858 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:08:56.272858 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 09:08:56.272858 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:08:56.272858 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:08:56.272858 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:08:56.272858 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:08:56.272858 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:08:56.272858 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:08:56.272858 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 24 09:08:56.272858 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:08:56.272858 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:08:56.272858 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:08:56.272858 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:08:56.272858 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:08:56.272858 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:08:56.273809 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:08:56.273809 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:08:56.273809 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 09:08:56.273809 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 09:08:56.273809 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 09:08:56.273809 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 09:08:56.273809 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 09:08:56.273809 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 09:08:56.273809 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 09:08:56.273809 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 24 09:08:56.274624 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 434/863] 10.222.0.16 () {56 vars in 1117 bytes} [Wed Aug 24 09:08:56 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:08:56.280722 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50699d0-5411-4715-aaf1-c6a37b5f2f92 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:56.398774 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50699d0-5411-4715-aaf1-c6a37b5f2f92 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:56.435191 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21e6589-fa89-43b8-9db0-68141a2ccaee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:56.435821 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21e6589-fa89-43b8-9db0-68141a2ccaee None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:56.436098 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21e6589-fa89-43b8-9db0-68141a2ccaee None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:56.436372 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21e6589-fa89-43b8-9db0-68141a2ccaee None admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:56.437095 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21e6589-fa89-43b8-9db0-68141a2ccaee None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:56.437394 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21e6589-fa89-43b8-9db0-68141a2ccaee None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:56.438809 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21e6589-fa89-43b8-9db0-68141a2ccaee None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:56.457337 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 430/864] 10.222.0.16 () {56 vars in 1149 bytes} [Wed Aug 24 09:08:56 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:56.463015 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2993b7d6-9131-43ac-9e84-9646c19db515 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:56.474061 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97d3c9d-b5f4-4e00-a716-9650c52e1bed None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:56.474346 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97d3c9d-b5f4-4e00-a716-9650c52e1bed None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:56.474619 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97d3c9d-b5f4-4e00-a716-9650c52e1bed None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:56.476184 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 431/865] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:08:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 09:08:56.482801 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecc89df-02be-46e0-bcf0-e796f433ff2e None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:56.483080 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecc89df-02be-46e0-bcf0-e796f433ff2e None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:56.483353 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecc89df-02be-46e0-bcf0-e796f433ff2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:56.502782 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-fecc89df-02be-46e0-bcf0-e796f433ff2e None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:08:56.504035 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-fecc89df-02be-46e0-bcf0-e796f433ff2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:56.559176 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:56.578219 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 432/866] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:08:56 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 09:08:56.582386 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2993b7d6-9131-43ac-9e84-9646c19db515 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:56.625817 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac08b5d1-2e53-47f6-a9ab-f6b7235d98c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:56.626780 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac08b5d1-2e53-47f6-a9ab-f6b7235d98c3 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:56.627261 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac08b5d1-2e53-47f6-a9ab-f6b7235d98c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:56.627783 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac08b5d1-2e53-47f6-a9ab-f6b7235d98c3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:56.647615 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:56.699259 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac08b5d1-2e53-47f6-a9ab-f6b7235d98c3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:56.701025 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac08b5d1-2e53-47f6-a9ab-f6b7235d98c3 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:56.808347 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 435/867] 10.222.0.16 () {58 vars in 1282 bytes} [Wed Aug 24 09:08:56 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 09:08:56.814536 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecc89df-02be-46e0-bcf0-e796f433ff2e None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:56.882193 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac08b5d1-2e53-47f6-a9ab-f6b7235d98c3 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:56.922086 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecc89df-02be-46e0-bcf0-e796f433ff2e None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:56.938237 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:56.958618 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56283a3a-eb47-439d-b94a-06fa6c65e53d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:56.959333 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56283a3a-eb47-439d-b94a-06fa6c65e53d None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:56.959746 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56283a3a-eb47-439d-b94a-06fa6c65e53d None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:56.960113 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56283a3a-eb47-439d-b94a-06fa6c65e53d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:56.970834 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56283a3a-eb47-439d-b94a-06fa6c65e53d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:56.972450 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56283a3a-eb47-439d-b94a-06fa6c65e53d None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:56.988170 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-56283a3a-eb47-439d-b94a-06fa6c65e53d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:08:56.988170 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:08:56.988170 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:08:56.988170 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:08:56.988170 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:08:56.988170 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:08:56.988170 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:08:56.988170 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:08:56.988170 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:08:56.988170 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:08:56.988170 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:08:56.988170 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:08:56.988170 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:08:56.988170 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:08:56.988170 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:08:56.988170 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:08:56.989291 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:08:56.989291 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:08:56.989291 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:08:56.989291 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:08:56.989291 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:08:56.989291 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:08:56.989291 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:08:56.989291 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Aug 24 09:08:56.989291 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:08:56.989291 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:08:56.989291 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:08:56.989291 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:08:56.989291 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:08:56.989291 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:08:56.989291 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:08:56.989291 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:08:56.989291 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:08:56.990189 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:08:56.990189 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:08:56.990189 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Aug 24 09:08:56.990373 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 433/868] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 24 09:08:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:08:56.996290 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56283a3a-eb47-439d-b94a-06fa6c65e53d None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:56.998225 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac08b5d1-2e53-47f6-a9ab-f6b7235d98c3 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:57.036437 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114396ce-2932-4fff-980e-9426a15641da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:57.037392 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114396ce-2932-4fff-980e-9426a15641da None admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:57.037854 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114396ce-2932-4fff-980e-9426a15641da None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:57.038305 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114396ce-2932-4fff-980e-9426a15641da None admin] PATH_INFO: `/v3/services` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:57.039292 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:57.040097 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114396ce-2932-4fff-980e-9426a15641da None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:57.041815 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114396ce-2932-4fff-980e-9426a15641da None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:57.106767 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56283a3a-eb47-439d-b94a-06fa6c65e53d None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:57.139526 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 436/869] 10.222.0.16 () {60 vars in 1141 bytes} [Wed Aug 24 09:08:56 2022] POST /identity/v3/services => generated 251 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:08:57.166499 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a11325f-65d1-4594-8a33-43c6f4ea79d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:57.167539 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a11325f-65d1-4594-8a33-43c6f4ea79d9 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:57.168121 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a11325f-65d1-4594-8a33-43c6f4ea79d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:57.168644 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a11325f-65d1-4594-8a33-43c6f4ea79d9 None admin] PATH_INFO: `/v3/domains` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:57.169546 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a11325f-65d1-4594-8a33-43c6f4ea79d9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:57.170169 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a11325f-65d1-4594-8a33-43c6f4ea79d9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:57.171922 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a11325f-65d1-4594-8a33-43c6f4ea79d9 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:57.187480 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 434/870] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Aug 24 09:08:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:57.194065 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114396ce-2932-4fff-980e-9426a15641da None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:57.225277 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88bb381-60e5-4f0f-81ec-edec438de517 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:57.225575 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88bb381-60e5-4f0f-81ec-edec438de517 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:57.225758 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88bb381-60e5-4f0f-81ec-edec438de517 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:57.227276 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 435/871] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:08:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:08:57.233694 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc8f9f0-99ba-4bc7-9c79-998f047d317f None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:57.234313 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc8f9f0-99ba-4bc7-9c79-998f047d317f None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:57.234782 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc8f9f0-99ba-4bc7-9c79-998f047d317f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:57.255434 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-3fc8f9f0-99ba-4bc7-9c79-998f047d317f None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:08:57.257023 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-3fc8f9f0-99ba-4bc7-9c79-998f047d317f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:57.334004 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114396ce-2932-4fff-980e-9426a15641da None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:57.347053 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 436/872] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:08:57 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 09:08:57.352223 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:57.377830 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3e52b6-45c8-4f39-b521-e0cb8290a6ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:57.378466 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3e52b6-45c8-4f39-b521-e0cb8290a6ea None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:57.378744 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3e52b6-45c8-4f39-b521-e0cb8290a6ea None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:57.379013 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3e52b6-45c8-4f39-b521-e0cb8290a6ea None admin] PATH_INFO: `/v3/projects/service` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:57.386293 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3e52b6-45c8-4f39-b521-e0cb8290a6ea None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:57.387820 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3e52b6-45c8-4f39-b521-e0cb8290a6ea None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:57.399808 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-3f3e52b6-45c8-4f39-b521-e0cb8290a6ea None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:08:57.399808 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:08:57.399808 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:08:57.399808 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:08:57.399808 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:08:57.399808 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:08:57.399808 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:08:57.399808 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:08:57.399808 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:08:57.399808 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:08:57.399808 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:08:57.399808 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:08:57.399808 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 09:08:57.399808 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 09:08:57.399808 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 09:08:57.401444 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 09:08:57.401444 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:08:57.401444 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:08:57.401444 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:08:57.401444 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:08:57.401444 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:08:57.401444 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:08:57.401444 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:08:57.401444 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 24 09:08:57.401444 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:08:57.401444 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:08:57.401444 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:08:57.401444 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:08:57.401444 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:08:57.401444 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:08:57.401444 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:08:57.401444 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:08:57.402563 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 09:08:57.402563 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 09:08:57.402563 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 09:08:57.402563 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 09:08:57.402563 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 09:08:57.402563 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 09:08:57.402563 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:08:57.402563 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 24 09:08:57.409186 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 437/873] 10.222.0.16 () {56 vars in 1111 bytes} [Wed Aug 24 09:08:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:08:57.414591 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc8f9f0-99ba-4bc7-9c79-998f047d317f None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:57.441724 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:57.519739 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc8f9f0-99ba-4bc7-9c79-998f047d317f None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:57.559613 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d92512a-d6f6-4ff0-965d-a1f0b60c04de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:57.560239 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d92512a-d6f6-4ff0-965d-a1f0b60c04de None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:57.560518 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d92512a-d6f6-4ff0-965d-a1f0b60c04de None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:57.560788 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d92512a-d6f6-4ff0-965d-a1f0b60c04de None admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:57.561546 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d92512a-d6f6-4ff0-965d-a1f0b60c04de None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:57.561853 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d92512a-d6f6-4ff0-965d-a1f0b60c04de None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:57.563281 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d92512a-d6f6-4ff0-965d-a1f0b60c04de None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:57.576066 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 437/874] 10.222.0.16 () {56 vars in 1148 bytes} [Wed Aug 24 09:08:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:57.582313 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3e52b6-45c8-4f39-b521-e0cb8290a6ea None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:57.674600 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d92512a-d6f6-4ff0-965d-a1f0b60c04de None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:57.694644 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3e52b6-45c8-4f39-b521-e0cb8290a6ea None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:57.749712 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f91b49-d755-4132-bb37-ce16dc60671f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:57.750332 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f91b49-d755-4132-bb37-ce16dc60671f None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:57.750612 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f91b49-d755-4132-bb37-ce16dc60671f None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:57.750885 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f91b49-d755-4132-bb37-ce16dc60671f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:57.751938 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f91b49-d755-4132-bb37-ce16dc60671f None admin] RBAC: Adding query filter params (role.id=67fd70a7d9794f7fb8faebffdec46dd2, scope.project.id=c467e88e2fb0491bacefcf6d2ce326d9, user.id=0489462287fc4aea8e8e720af24e1ef8) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:57.752248 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f91b49-d755-4132-bb37-ce16dc60671f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:57.753973 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f91b49-d755-4132-bb37-ce16dc60671f None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:57.771693 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 438/875] 10.222.0.16 () {56 vars in 1374 bytes} [Wed Aug 24 09:08:57 2022] GET /identity/v3/role_assignments?user.id=0489462287fc4aea8e8e720af24e1ef8&scope.project.id=c467e88e2fb0491bacefcf6d2ce326d9&role.id=67fd70a7d9794f7fb8faebffdec46dd2 => generated 608 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:57.797085 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:57.814700 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d92512a-d6f6-4ff0-965d-a1f0b60c04de None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:57.856171 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd06f7d-2bf6-4a60-8d6b-b5011360cdd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:57.856790 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd06f7d-2bf6-4a60-8d6b-b5011360cdd1 None admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:57.857121 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd06f7d-2bf6-4a60-8d6b-b5011360cdd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:57.857401 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd06f7d-2bf6-4a60-8d6b-b5011360cdd1 None admin] PATH_INFO: `/v3/services` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:57.858066 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd06f7d-2bf6-4a60-8d6b-b5011360cdd1 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:57.859433 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd06f7d-2bf6-4a60-8d6b-b5011360cdd1 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:57.891853 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82caa0ce-7e61-4833-b118-cc7aeed8139c None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:57.892381 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82caa0ce-7e61-4833-b118-cc7aeed8139c None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:57.892459 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82caa0ce-7e61-4833-b118-cc7aeed8139c None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:57.893917 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 439/876] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:08:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 09:08:57.899960 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec2324b-9f7f-4a39-b063-3ae49f06b4bd None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:57.900814 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec2324b-9f7f-4a39-b063-3ae49f06b4bd None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:57.901108 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec2324b-9f7f-4a39-b063-3ae49f06b4bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:57.920903 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-fec2324b-9f7f-4a39-b063-3ae49f06b4bd None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:08:57.922111 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-fec2324b-9f7f-4a39-b063-3ae49f06b4bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:57.952931 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:57.966652 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 438/877] 10.222.0.16 () {60 vars in 1141 bytes} [Wed Aug 24 09:08:57 2022] POST /identity/v3/services => generated 269 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 09:08:58.006977 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 440/878] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:08:57 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 09:08:58.276036 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd06f7d-2bf6-4a60-8d6b-b5011360cdd1 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:58.386741 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd06f7d-2bf6-4a60-8d6b-b5011360cdd1 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:58.423065 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859d3ecb-8e75-484f-9ba9-1347c1c9e484 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:58.423731 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859d3ecb-8e75-484f-9ba9-1347c1c9e484 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:58.424013 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859d3ecb-8e75-484f-9ba9-1347c1c9e484 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:58.424288 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859d3ecb-8e75-484f-9ba9-1347c1c9e484 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:58.424874 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859d3ecb-8e75-484f-9ba9-1347c1c9e484 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:58.426263 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859d3ecb-8e75-484f-9ba9-1347c1c9e484 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:58.435611 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-859d3ecb-8e75-484f-9ba9-1347c1c9e484 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 09:08:58.435611 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:08:58.435611 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 09:08:58.435611 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 09:08:58.435611 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 09:08:58.435611 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 09:08:58.435611 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 09:08:58.435611 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:08:58.435611 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 09:08:58.435611 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Aug 24 09:08:58.435611 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 09:08:58.435611 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Aug 24 09:08:58.435611 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:08:58.435611 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:08:58.435611 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:08:58.435611 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:08:58.435611 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:08:58.436594 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:08:58.436594 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:08:58.436594 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:08:58.436594 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:08:58.436594 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:08:58.436594 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:08:58.436594 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 09:08:58.436594 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 09:08:58.436594 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 09:08:58.436594 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 09:08:58.436594 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:08:58.436594 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:08:58.436594 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:08:58.436594 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:08:58.436594 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:08:58.436594 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:08:58.437457 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:08:58.437457 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Aug 24 09:08:58.437457 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:08:58.437457 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:08:58.437457 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:08:58.437457 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:08:58.437457 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:08:58.437457 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:08:58.437457 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:08:58.437457 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:08:58.437457 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 09:08:58.437457 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:08:58.437457 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 09:08:58.437457 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Aug 24 09:08:58.437457 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 439/879] 10.222.0.16 () {56 vars in 1126 bytes} [Wed Aug 24 09:08:58 2022] GET /identity/v3/services/object-store => generated 93 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:08:58.442817 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec2324b-9f7f-4a39-b063-3ae49f06b4bd None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:58.542687 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec2324b-9f7f-4a39-b063-3ae49f06b4bd None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:58.576555 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6f187a-9612-4730-9a35-65e8ee0ebce8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:58.577243 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6f187a-9612-4730-9a35-65e8ee0ebce8 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:58.577519 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6f187a-9612-4730-9a35-65e8ee0ebce8 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:58.577788 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6f187a-9612-4730-9a35-65e8ee0ebce8 None admin] PATH_INFO: `/v3/services` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:58.578414 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6f187a-9612-4730-9a35-65e8ee0ebce8 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:58.579025 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6f187a-9612-4730-9a35-65e8ee0ebce8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:58.580402 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6f187a-9612-4730-9a35-65e8ee0ebce8 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:58.585241 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 441/880] 10.222.0.16 () {56 vars in 1122 bytes} [Wed Aug 24 09:08:58 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:58.590675 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859d3ecb-8e75-484f-9ba9-1347c1c9e484 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:58.602172 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:58.696918 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859d3ecb-8e75-484f-9ba9-1347c1c9e484 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:58.733159 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5375882-dce2-41c8-9d5b-8b46bf9a6dde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:58.733771 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5375882-dce2-41c8-9d5b-8b46bf9a6dde None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:58.734053 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5375882-dce2-41c8-9d5b-8b46bf9a6dde None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:58.734326 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5375882-dce2-41c8-9d5b-8b46bf9a6dde None admin] PATH_INFO: `/v3/services` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:58.734957 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5375882-dce2-41c8-9d5b-8b46bf9a6dde None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:58.735263 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5375882-dce2-41c8-9d5b-8b46bf9a6dde None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:58.736662 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5375882-dce2-41c8-9d5b-8b46bf9a6dde None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:58.741618 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 440/881] 10.222.0.16 () {56 vars in 1122 bytes} [Wed Aug 24 09:08:58 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:59.229254 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac49391-3213-499c-93d8-945b160d8998 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:59.229799 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac49391-3213-499c-93d8-945b160d8998 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:59.229799 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac49391-3213-499c-93d8-945b160d8998 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:59.231378 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 442/882] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:08:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:08:59.236698 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518c73fe-57e7-41ff-9d1f-ee32e5cea72b None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:59.237132 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518c73fe-57e7-41ff-9d1f-ee32e5cea72b None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:59.237277 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518c73fe-57e7-41ff-9d1f-ee32e5cea72b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:59.255960 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-518c73fe-57e7-41ff-9d1f-ee32e5cea72b None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:08:59.257160 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-518c73fe-57e7-41ff-9d1f-ee32e5cea72b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:59.338595 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 441/883] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:08:59 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 09:08:59.603071 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac49391-3213-499c-93d8-945b160d8998 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:59.629975 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:59.711573 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac49391-3213-499c-93d8-945b160d8998 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:59.747903 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ffa1ec-74d1-47de-8327-9ab7da299170 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:59.748545 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ffa1ec-74d1-47de-8327-9ab7da299170 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:59.748824 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ffa1ec-74d1-47de-8327-9ab7da299170 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:59.749096 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ffa1ec-74d1-47de-8327-9ab7da299170 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:59.749685 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ffa1ec-74d1-47de-8327-9ab7da299170 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:59.751053 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ffa1ec-74d1-47de-8327-9ab7da299170 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:59.760077 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-45ffa1ec-74d1-47de-8327-9ab7da299170 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 09:08:59.760077 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:08:59.760077 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 09:08:59.760077 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 09:08:59.760077 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 09:08:59.760077 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 09:08:59.760077 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 09:08:59.760077 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:08:59.760077 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 09:08:59.760077 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 24 09:08:59.760077 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 09:08:59.760077 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 24 09:08:59.760077 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:08:59.760077 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:08:59.760077 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:08:59.760077 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:08:59.760077 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:08:59.761057 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:08:59.761057 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:08:59.761057 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:08:59.761057 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:08:59.761057 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:08:59.761057 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:08:59.761057 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 09:08:59.761057 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 09:08:59.761057 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 09:08:59.761057 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 09:08:59.761057 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:08:59.761057 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:08:59.761057 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:08:59.761057 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:08:59.761057 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:08:59.761057 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:08:59.761910 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:08:59.761910 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 24 09:08:59.761910 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:08:59.761910 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:08:59.761910 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:08:59.761910 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:08:59.761910 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:08:59.761910 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:08:59.761910 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:08:59.761910 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:08:59.761910 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 09:08:59.761910 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:08:59.761910 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 09:08:59.761910 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 24 09:08:59.761910 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 443/884] 10.222.0.16 () {56 vars in 1114 bytes} [Wed Aug 24 09:08:59 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:08:59.767215 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518c73fe-57e7-41ff-9d1f-ee32e5cea72b None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:59.769722 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:08:59.880366 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518c73fe-57e7-41ff-9d1f-ee32e5cea72b None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:08:59.881074 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a90be9-b4e9-4cb6-a21e-caf962e14b15 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:59.881569 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a90be9-b4e9-4cb6-a21e-caf962e14b15 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:59.882045 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a90be9-b4e9-4cb6-a21e-caf962e14b15 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:59.883864 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 444/885] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:08:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 09:08:59.889790 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824089a4-7e10-4792-bcfb-8dc1bf7c890a None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:59.890319 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824089a4-7e10-4792-bcfb-8dc1bf7c890a None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:59.890826 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824089a4-7e10-4792-bcfb-8dc1bf7c890a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:59.910011 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-824089a4-7e10-4792-bcfb-8dc1bf7c890a None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:08:59.911697 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-824089a4-7e10-4792-bcfb-8dc1bf7c890a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:08:59.923423 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a45f9d0-a1b4-46a5-a34e-a48df19817bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:08:59.924419 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a45f9d0-a1b4-46a5-a34e-a48df19817bb None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:59.924903 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a45f9d0-a1b4-46a5-a34e-a48df19817bb None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:59.925457 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a45f9d0-a1b4-46a5-a34e-a48df19817bb None admin] PATH_INFO: `/v3/services` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:59.926287 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a45f9d0-a1b4-46a5-a34e-a48df19817bb None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:08:59.926811 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a45f9d0-a1b4-46a5-a34e-a48df19817bb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:08:59.928500 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a45f9d0-a1b4-46a5-a34e-a48df19817bb None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:08:59.934582 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 442/886] 10.222.0.16 () {56 vars in 1114 bytes} [Wed Aug 24 09:08:59 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:08:59.936022 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4f849e-8c98-4ac8-a597-13659dc3f5b6 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:59.936622 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4f849e-8c98-4ac8-a597-13659dc3f5b6 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:59.937126 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4f849e-8c98-4ac8-a597-13659dc3f5b6 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:08:59.938518 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 443/887] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:08:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:08:59.939939 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4f849e-8c98-4ac8-a597-13659dc3f5b6 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:08:59.987384 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 445/888] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:08:59 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 09:08:59.988737 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a91526b-adfb-4ca2-8db3-bb45f725b454 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:08:59.989022 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a91526b-adfb-4ca2-8db3-bb45f725b454 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:08:59.989295 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a91526b-adfb-4ca2-8db3-bb45f725b454 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:00.014117 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:00.014596 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-0a91526b-adfb-4ca2-8db3-bb45f725b454 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:09:00.016031 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-0a91526b-adfb-4ca2-8db3-bb45f725b454 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:09:00.030550 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:00.052380 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4f849e-8c98-4ac8-a597-13659dc3f5b6 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:00.094859 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56e1a4a-af6c-42ca-8a07-0d5dde64574c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:00.095471 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56e1a4a-af6c-42ca-8a07-0d5dde64574c None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:00.095778 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56e1a4a-af6c-42ca-8a07-0d5dde64574c None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:00.096057 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56e1a4a-af6c-42ca-8a07-0d5dde64574c None admin] PATH_INFO: `/v3/services` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:00.096689 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56e1a4a-af6c-42ca-8a07-0d5dde64574c None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:09:00.096993 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56e1a4a-af6c-42ca-8a07-0d5dde64574c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:00.098344 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56e1a4a-af6c-42ca-8a07-0d5dde64574c None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:00.105107 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 444/889] 10.222.0.16 () {56 vars in 1114 bytes} [Wed Aug 24 09:08:59 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:09:00.111168 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56e1a4a-af6c-42ca-8a07-0d5dde64574c None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:00.119956 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 446/890] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:08:59 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 09:09:00.225216 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56e1a4a-af6c-42ca-8a07-0d5dde64574c None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:00.256127 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:00.262080 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e861b232-a575-4b4a-b50b-4050070390b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:00.262699 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e861b232-a575-4b4a-b50b-4050070390b1 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:00.263027 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e861b232-a575-4b4a-b50b-4050070390b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:00.263303 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e861b232-a575-4b4a-b50b-4050070390b1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:00.264035 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e861b232-a575-4b4a-b50b-4050070390b1 None admin] RBAC: Adding query filter params (interface=public, service_id=9fb3c0d8ded142e78b35a665e35bd52a, region_id=RegionOne) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:09:00.264342 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e861b232-a575-4b4a-b50b-4050070390b1 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:00.265753 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e861b232-a575-4b4a-b50b-4050070390b1 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:00.268731 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a91526b-adfb-4ca2-8db3-bb45f725b454 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:00.271670 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 445/891] 10.222.0.16 () {56 vars in 1251 bytes} [Wed Aug 24 09:09:00 2022] GET /identity/v3/endpoints?service_id=9fb3c0d8ded142e78b35a665e35bd52a&interface=public®ion_id=RegionOne => generated 194 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:09:00.277799 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e861b232-a575-4b4a-b50b-4050070390b1 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:00.394529 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e861b232-a575-4b4a-b50b-4050070390b1 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:00.401073 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a91526b-adfb-4ca2-8db3-bb45f725b454 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:00.433780 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfd55b2-8e54-43a9-a120-1ea175dd0a59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:00.434679 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfd55b2-8e54-43a9-a120-1ea175dd0a59 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:00.434975 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfd55b2-8e54-43a9-a120-1ea175dd0a59 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:00.435247 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfd55b2-8e54-43a9-a120-1ea175dd0a59 None admin] PATH_INFO: `/v3/services` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:00.435897 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfd55b2-8e54-43a9-a120-1ea175dd0a59 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:00.437270 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfd55b2-8e54-43a9-a120-1ea175dd0a59 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:00.443668 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61ed5f4-2d7c-4b3a-a7ab-c42ea77db17f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:00.444316 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61ed5f4-2d7c-4b3a-a7ab-c42ea77db17f None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:00.444599 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61ed5f4-2d7c-4b3a-a7ab-c42ea77db17f None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:00.444874 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61ed5f4-2d7c-4b3a-a7ab-c42ea77db17f None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:00.456068 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 446/892] 10.222.0.16 () {56 vars in 1087 bytes} [Wed Aug 24 09:09:00 2022] GET /identity/v3/services => generated 1268 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:09:00.456068 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfd55b2-8e54-43a9-a120-1ea175dd0a59 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:00.467986 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61ed5f4-2d7c-4b3a-a7ab-c42ea77db17f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:09:00.473280 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-e61ed5f4-2d7c-4b3a-a7ab-c42ea77db17f None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 09:09:00.473280 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:09:00.473280 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:09:00.473280 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:09:00.473280 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:09:00.473280 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:09:00.473280 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:09:00.473280 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:09:00.473280 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:09:00.473280 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:09:00.473280 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:09:00.473280 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:09:00.473280 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:09:00.473280 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:09:00.473280 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:09:00.473280 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:09:00.474323 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:09:00.474323 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:09:00.474323 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:09:00.474323 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:09:00.474323 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:09:00.474323 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:09:00.474323 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:09:00.474323 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:09:00.474323 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:09:00.474323 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:09:00.474323 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:09:00.474323 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 24 09:09:00.474323 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:09:00.474323 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:09:00.474323 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:09:00.474323 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:09:00.475184 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:09:00.475184 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:09:00.475184 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:09:00.475184 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:09:00.475184 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:09:00.475184 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:09:00.475184 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:09:00.475184 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:09:00.475184 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:09:00.475184 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:09:00.475184 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 09:09:00.475184 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 24 09:09:00.475184 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 447/893] 10.222.0.16 () {56 vars in 1120 bytes} [Wed Aug 24 09:09:00 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 09:09:00.481205 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61ed5f4-2d7c-4b3a-a7ab-c42ea77db17f None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:00.496375 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:00.519085 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:00.565846 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfd55b2-8e54-43a9-a120-1ea175dd0a59 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:00.605295 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd8d7b1-ce17-4905-bf1f-6da4cdf25906 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:00.605940 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd8d7b1-ce17-4905-bf1f-6da4cdf25906 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:00.606224 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd8d7b1-ce17-4905-bf1f-6da4cdf25906 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:00.606502 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd8d7b1-ce17-4905-bf1f-6da4cdf25906 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:00.607097 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd8d7b1-ce17-4905-bf1f-6da4cdf25906 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:00.608518 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd8d7b1-ce17-4905-bf1f-6da4cdf25906 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:00.618232 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-2bd8d7b1-ce17-4905-bf1f-6da4cdf25906 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 09:09:00.618232 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:09:00.618232 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 09:09:00.618232 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 09:09:00.618232 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 09:09:00.618232 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 09:09:00.618232 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 09:09:00.618232 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:09:00.618232 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 09:09:00.618232 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Aug 24 09:09:00.618232 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 09:09:00.618232 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Aug 24 09:09:00.618232 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:09:00.618232 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:09:00.618232 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:09:00.618232 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:09:00.619302 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:09:00.619302 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:09:00.619302 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:09:00.619302 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:09:00.619302 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:09:00.619302 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:09:00.619302 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:09:00.619302 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 09:09:00.619302 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 09:09:00.619302 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 09:09:00.619302 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 09:09:00.619302 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:09:00.619302 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:09:00.619302 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:09:00.619302 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:09:00.619302 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:09:00.619302 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:09:00.620243 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:09:00.620243 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Aug 24 09:09:00.620243 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:09:00.620243 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:09:00.620243 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:09:00.620243 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:09:00.620243 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:09:00.620243 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:09:00.620243 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:09:00.620243 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:09:00.620243 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 09:09:00.620243 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:09:00.620243 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 09:09:00.620243 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Aug 24 09:09:00.620243 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 447/894] 10.222.0.16 () {56 vars in 1132 bytes} [Wed Aug 24 09:09:00 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:09:00.626136 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd8d7b1-ce17-4905-bf1f-6da4cdf25906 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:00.628755 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61ed5f4-2d7c-4b3a-a7ab-c42ea77db17f None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:00.677021 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d00a49c-c098-40c1-b864-04186450a8b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:00.677629 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d00a49c-c098-40c1-b864-04186450a8b3 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:00.677911 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d00a49c-c098-40c1-b864-04186450a8b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:00.678183 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d00a49c-c098-40c1-b864-04186450a8b3 None admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:00.678851 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d00a49c-c098-40c1-b864-04186450a8b3 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:09:00.679157 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d00a49c-c098-40c1-b864-04186450a8b3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:00.680605 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d00a49c-c098-40c1-b864-04186450a8b3 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:00.688907 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 448/895] 10.222.0.16 () {56 vars in 1115 bytes} [Wed Aug 24 09:09:00 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:09:00.690176 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4442b0d-99c0-4d91-8b1c-ac9a71e3e7f1 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:00.690451 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4442b0d-99c0-4d91-8b1c-ac9a71e3e7f1 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:00.690719 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4442b0d-99c0-4d91-8b1c-ac9a71e3e7f1 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:00.692191 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 449/896] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:09:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:09:00.700860 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0d4276-294d-4535-bc2e-4d2140dfa279 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:00.701146 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0d4276-294d-4535-bc2e-4d2140dfa279 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:00.701419 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0d4276-294d-4535-bc2e-4d2140dfa279 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:00.721117 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-8a0d4276-294d-4535-bc2e-4d2140dfa279 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:09:00.722317 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-8a0d4276-294d-4535-bc2e-4d2140dfa279 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:09:00.753209 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd8d7b1-ce17-4905-bf1f-6da4cdf25906 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:00.778246 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:00.789916 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:00.791911 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25210b85-e461-4c05-85e8-96fc07d760bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:00.792523 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25210b85-e461-4c05-85e8-96fc07d760bc None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:00.792803 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25210b85-e461-4c05-85e8-96fc07d760bc None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:00.793132 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25210b85-e461-4c05-85e8-96fc07d760bc None admin] PATH_INFO: `/v3/services` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:00.793843 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25210b85-e461-4c05-85e8-96fc07d760bc None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:09:00.794153 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25210b85-e461-4c05-85e8-96fc07d760bc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:00.799638 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 450/897] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:09:00 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 09:09:00.799793 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25210b85-e461-4c05-85e8-96fc07d760bc None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:00.799873 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0d4276-294d-4535-bc2e-4d2140dfa279 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:00.810436 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 448/898] 10.222.0.16 () {56 vars in 1126 bytes} [Wed Aug 24 09:09:00 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:09:00.815836 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25210b85-e461-4c05-85e8-96fc07d760bc None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:00.909099 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0d4276-294d-4535-bc2e-4d2140dfa279 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:00.947403 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942d03c4-e371-4afb-b466-2d6e07c1c5e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:00.948115 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942d03c4-e371-4afb-b466-2d6e07c1c5e6 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:00.948405 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942d03c4-e371-4afb-b466-2d6e07c1c5e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:00.948675 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942d03c4-e371-4afb-b466-2d6e07c1c5e6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:00.950608 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25210b85-e461-4c05-85e8-96fc07d760bc None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:00.959422 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942d03c4-e371-4afb-b466-2d6e07c1c5e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:00.960906 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942d03c4-e371-4afb-b466-2d6e07c1c5e6 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:00.976552 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-942d03c4-e371-4afb-b466-2d6e07c1c5e6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:09:00.976552 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:09:00.976552 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:09:00.976552 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:09:00.976552 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:09:00.976552 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:09:00.976552 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:09:00.976552 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:09:00.976552 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:09:00.976552 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:09:00.976552 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:09:00.976552 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:09:00.976552 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:09:00.976552 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:09:00.976552 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:09:00.976552 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:09:00.977550 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:09:00.977550 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:09:00.977550 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:09:00.977550 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:09:00.977550 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:09:00.977550 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:09:00.977550 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:09:00.977550 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 24 09:09:00.977550 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:09:00.977550 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:09:00.977550 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:09:00.977550 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:09:00.977550 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:09:00.977550 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:09:00.977550 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:09:00.977550 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:09:00.977550 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:09:00.979238 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:09:00.979238 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:09:00.979238 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 24 09:09:00.979238 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 451/899] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 24 09:09:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:09:00.984250 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942d03c4-e371-4afb-b466-2d6e07c1c5e6 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:00.997278 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468a1b47-51b8-475c-a7c0-e53678507b8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:00.998532 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468a1b47-51b8-475c-a7c0-e53678507b8e None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:00.999089 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468a1b47-51b8-475c-a7c0-e53678507b8e None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:00.999527 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468a1b47-51b8-475c-a7c0-e53678507b8e None admin] PATH_INFO: `/v3/services` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:01.000342 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468a1b47-51b8-475c-a7c0-e53678507b8e None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:09:01.001019 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468a1b47-51b8-475c-a7c0-e53678507b8e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:01.003265 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468a1b47-51b8-475c-a7c0-e53678507b8e None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:01.010218 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 449/900] 10.222.0.16 () {56 vars in 1126 bytes} [Wed Aug 24 09:09:00 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:09:01.015538 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468a1b47-51b8-475c-a7c0-e53678507b8e None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:01.053046 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:01.084634 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:01.109828 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942d03c4-e371-4afb-b466-2d6e07c1c5e6 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:01.137172 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468a1b47-51b8-475c-a7c0-e53678507b8e None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:01.146426 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704aada2-e092-461a-8bc3-91d5b6080fe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:01.147265 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704aada2-e092-461a-8bc3-91d5b6080fe1 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:01.147378 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704aada2-e092-461a-8bc3-91d5b6080fe1 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:01.147576 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704aada2-e092-461a-8bc3-91d5b6080fe1 None admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:01.148217 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704aada2-e092-461a-8bc3-91d5b6080fe1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:09:01.148521 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704aada2-e092-461a-8bc3-91d5b6080fe1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:01.150379 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704aada2-e092-461a-8bc3-91d5b6080fe1 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:01.167419 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 452/901] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Aug 24 09:09:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:09:01.167419 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704aada2-e092-461a-8bc3-91d5b6080fe1 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:01.181471 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ac1da4-0e2f-4a30-bce4-7379205e9a54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:01.182441 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ac1da4-0e2f-4a30-bce4-7379205e9a54 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:01.182441 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ac1da4-0e2f-4a30-bce4-7379205e9a54 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:01.182700 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ac1da4-0e2f-4a30-bce4-7379205e9a54 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:01.183402 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ac1da4-0e2f-4a30-bce4-7379205e9a54 None admin] RBAC: Adding query filter params (interface=public, service_id=49d2b3c0b2704733a945fedf78ea6c9f, region_id=RegionOne) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:09:01.183760 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ac1da4-0e2f-4a30-bce4-7379205e9a54 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:01.185126 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ac1da4-0e2f-4a30-bce4-7379205e9a54 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:01.194717 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 450/902] 10.222.0.16 () {56 vars in 1251 bytes} [Wed Aug 24 09:09:01 2022] GET /identity/v3/endpoints?service_id=49d2b3c0b2704733a945fedf78ea6c9f&interface=public®ion_id=RegionOne => generated 194 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:09:01.194717 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ac1da4-0e2f-4a30-bce4-7379205e9a54 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:01.277680 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704aada2-e092-461a-8bc3-91d5b6080fe1 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:01.292765 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:01.299841 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ac1da4-0e2f-4a30-bce4-7379205e9a54 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:01.315257 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:01.319922 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20d6fac-d909-48e2-a7f3-11d278966b69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:01.320538 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20d6fac-d909-48e2-a7f3-11d278966b69 None admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:01.320816 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20d6fac-d909-48e2-a7f3-11d278966b69 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:01.321089 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20d6fac-d909-48e2-a7f3-11d278966b69 None admin] PATH_INFO: `/v3/services` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:01.321758 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20d6fac-d909-48e2-a7f3-11d278966b69 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:01.323115 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20d6fac-d909-48e2-a7f3-11d278966b69 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:01.340460 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16321950-ad97-44fa-8647-f393015a5625 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:01.341088 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16321950-ad97-44fa-8647-f393015a5625 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:01.341364 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16321950-ad97-44fa-8647-f393015a5625 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:01.341657 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16321950-ad97-44fa-8647-f393015a5625 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:01.349093 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16321950-ad97-44fa-8647-f393015a5625 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:01.350511 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16321950-ad97-44fa-8647-f393015a5625 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:01.362701 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-16321950-ad97-44fa-8647-f393015a5625 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 09:09:01.362701 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:09:01.362701 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:09:01.362701 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:09:01.362701 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:09:01.362701 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:09:01.362701 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:09:01.362701 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:09:01.362701 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:09:01.362701 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:09:01.362701 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:09:01.362701 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:09:01.362701 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 09:09:01.362701 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 09:09:01.362701 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 09:09:01.362701 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 09:09:01.371006 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:09:01.371006 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:09:01.371006 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 09:09:01.371006 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:09:01.371006 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 09:09:01.371006 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:09:01.371006 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:09:01.371006 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:09:01.371006 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:09:01.371006 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:09:01.371006 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:09:01.371006 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Aug 24 09:09:01.371006 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:09:01.371006 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:09:01.371006 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:09:01.371006 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:09:01.371006 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:09:01.371006 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:09:01.372290 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:09:01.372290 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:09:01.372290 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 09:09:01.372290 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 09:09:01.372290 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 09:09:01.372290 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 09:09:01.372290 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 09:09:01.372290 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 09:09:01.372290 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 09:09:01.372290 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Aug 24 09:09:01.372290 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 451/903] 10.222.0.16 () {56 vars in 1117 bytes} [Wed Aug 24 09:09:01 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:09:01.372290 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16321950-ad97-44fa-8647-f393015a5625 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:01.395915 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 453/904] 10.222.0.16 () {60 vars in 1141 bytes} [Wed Aug 24 09:09:01 2022] POST /identity/v3/services => generated 241 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:09:01.396663 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20d6fac-d909-48e2-a7f3-11d278966b69 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:01.506860 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16321950-ad97-44fa-8647-f393015a5625 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:01.528815 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20d6fac-d909-48e2-a7f3-11d278966b69 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:01.546802 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b766c34d-7f3a-4ba2-8f7f-b6bbdb6b1eff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:01.547863 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b766c34d-7f3a-4ba2-8f7f-b6bbdb6b1eff None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:01.548212 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b766c34d-7f3a-4ba2-8f7f-b6bbdb6b1eff None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:01.548481 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b766c34d-7f3a-4ba2-8f7f-b6bbdb6b1eff None admin] PATH_INFO: `/v3/services` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:01.549095 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b766c34d-7f3a-4ba2-8f7f-b6bbdb6b1eff None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:01.551231 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b766c34d-7f3a-4ba2-8f7f-b6bbdb6b1eff None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:01.558576 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 452/905] 10.222.0.16 () {56 vars in 1087 bytes} [Wed Aug 24 09:09:01 2022] GET /identity/v3/services => generated 1497 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:09:01.571068 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3045ab95-7371-4c9d-94ef-1961231d9877 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:01.571775 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3045ab95-7371-4c9d-94ef-1961231d9877 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:01.572060 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3045ab95-7371-4c9d-94ef-1961231d9877 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:01.572337 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3045ab95-7371-4c9d-94ef-1961231d9877 None admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:01.573092 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3045ab95-7371-4c9d-94ef-1961231d9877 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:09:01.573398 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3045ab95-7371-4c9d-94ef-1961231d9877 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:01.574879 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3045ab95-7371-4c9d-94ef-1961231d9877 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:01.598368 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 454/906] 10.222.0.16 () {56 vars in 1149 bytes} [Wed Aug 24 09:09:01 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:09:01.605267 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b766c34d-7f3a-4ba2-8f7f-b6bbdb6b1eff None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:01.657147 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:01.723752 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b766c34d-7f3a-4ba2-8f7f-b6bbdb6b1eff None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:01.760753 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4978eda-36e4-49ef-88ca-76d8f6eec115 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:01.761369 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4978eda-36e4-49ef-88ca-76d8f6eec115 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:01.761645 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4978eda-36e4-49ef-88ca-76d8f6eec115 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:01.761916 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4978eda-36e4-49ef-88ca-76d8f6eec115 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:01.832389 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4978eda-36e4-49ef-88ca-76d8f6eec115 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:01.833819 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4978eda-36e4-49ef-88ca-76d8f6eec115 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:01.875890 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:01.939902 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 453/907] 10.222.0.16 () {58 vars in 1282 bytes} [Wed Aug 24 09:09:01 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 09:09:01.945883 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3045ab95-7371-4c9d-94ef-1961231d9877 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:01.976243 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:02.058429 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3045ab95-7371-4c9d-94ef-1961231d9877 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:02.096729 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7378321-1c0a-4a9d-b02e-6751a7d10c8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:02.097300 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7378321-1c0a-4a9d-b02e-6751a7d10c8a None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:02.097575 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7378321-1c0a-4a9d-b02e-6751a7d10c8a None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:02.097850 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7378321-1c0a-4a9d-b02e-6751a7d10c8a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:02.109268 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7378321-1c0a-4a9d-b02e-6751a7d10c8a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:02.110710 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7378321-1c0a-4a9d-b02e-6751a7d10c8a None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:02.125633 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-f7378321-1c0a-4a9d-b02e-6751a7d10c8a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:09:02.125633 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:09:02.125633 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:09:02.125633 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:09:02.125633 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:09:02.125633 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:09:02.125633 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:09:02.125633 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:09:02.125633 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:09:02.125633 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:09:02.125633 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:09:02.125633 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:09:02.125633 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:09:02.125633 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:09:02.125633 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:09:02.125633 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:09:02.126631 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:09:02.126631 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:09:02.126631 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:09:02.126631 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:09:02.126631 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:09:02.126631 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:09:02.126631 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:09:02.126631 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 24 09:09:02.126631 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:09:02.126631 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:09:02.126631 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:09:02.126631 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:09:02.126631 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:09:02.126631 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:09:02.126631 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:09:02.126631 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:09:02.126631 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:09:02.128035 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:09:02.128035 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:09:02.128035 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 24 09:09:02.128035 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 455/908] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 24 09:09:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:09:02.132468 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4978eda-36e4-49ef-88ca-76d8f6eec115 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:02.246396 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4978eda-36e4-49ef-88ca-76d8f6eec115 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:02.283459 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9af0ec9-b49b-482b-94c0-fedc251e7f67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:02.284097 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9af0ec9-b49b-482b-94c0-fedc251e7f67 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:02.284374 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9af0ec9-b49b-482b-94c0-fedc251e7f67 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:02.284642 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9af0ec9-b49b-482b-94c0-fedc251e7f67 None admin] PATH_INFO: `/v3/domains` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:02.285282 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9af0ec9-b49b-482b-94c0-fedc251e7f67 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:09:02.285581 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9af0ec9-b49b-482b-94c0-fedc251e7f67 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:02.286948 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9af0ec9-b49b-482b-94c0-fedc251e7f67 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:02.293938 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:02.299329 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 454/909] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Aug 24 09:09:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:09:02.304776 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7378321-1c0a-4a9d-b02e-6751a7d10c8a None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:02.392062 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:02.419772 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7378321-1c0a-4a9d-b02e-6751a7d10c8a None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:02.458829 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bdf1b5-c6ec-428e-852d-590d4125d7ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:02.459469 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bdf1b5-c6ec-428e-852d-590d4125d7ed None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:02.459833 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bdf1b5-c6ec-428e-852d-590d4125d7ed None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:02.460106 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bdf1b5-c6ec-428e-852d-590d4125d7ed None admin] PATH_INFO: `/v3/projects/service` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:02.461258 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047900f8-6a17-4036-8121-ea63e3102e5d None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:02.461695 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047900f8-6a17-4036-8121-ea63e3102e5d None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:02.461855 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047900f8-6a17-4036-8121-ea63e3102e5d None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:02.463954 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 455/910] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:09:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:09:02.468385 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0bdf1b5-c6ec-428e-852d-590d4125d7ed None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:02.469230 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692780c9-03b7-4c4b-8b0d-290456de4416 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:02.469510 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692780c9-03b7-4c4b-8b0d-290456de4416 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:02.469783 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692780c9-03b7-4c4b-8b0d-290456de4416 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:02.470805 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0bdf1b5-c6ec-428e-852d-590d4125d7ed None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:02.485776 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-a0bdf1b5-c6ec-428e-852d-590d4125d7ed None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:09:02.485776 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:09:02.485776 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:09:02.485776 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:09:02.485776 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:09:02.485776 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:09:02.485776 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:09:02.485776 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:09:02.485776 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:09:02.485776 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:09:02.485776 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:09:02.485776 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:09:02.485776 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 09:09:02.485776 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 09:09:02.485776 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 09:09:02.486894 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 09:09:02.486894 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:09:02.486894 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:09:02.486894 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:09:02.486894 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:09:02.486894 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:09:02.486894 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:09:02.486894 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:09:02.486894 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 24 09:09:02.486894 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:09:02.486894 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:09:02.486894 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:09:02.486894 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:09:02.486894 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:09:02.486894 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:09:02.486894 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:09:02.486894 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:09:02.488097 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 09:09:02.488097 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 09:09:02.488097 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 09:09:02.488097 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 09:09:02.488097 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 09:09:02.488097 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 09:09:02.488097 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:09:02.488097 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 24 09:09:02.488097 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 456/911] 10.222.0.16 () {56 vars in 1111 bytes} [Wed Aug 24 09:09:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:09:02.494280 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-692780c9-03b7-4c4b-8b0d-290456de4416 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:09:02.495458 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-692780c9-03b7-4c4b-8b0d-290456de4416 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:09:02.500505 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bdf1b5-c6ec-428e-852d-590d4125d7ed None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:02.578159 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 456/912] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:09:02 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 09:09:02.617118 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bdf1b5-c6ec-428e-852d-590d4125d7ed None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:02.656276 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91527e9-f45d-4ba6-b77d-504eca38c5c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:02.656959 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91527e9-f45d-4ba6-b77d-504eca38c5c6 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:02.657339 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91527e9-f45d-4ba6-b77d-504eca38c5c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:02.657710 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91527e9-f45d-4ba6-b77d-504eca38c5c6 None admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:02.658511 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91527e9-f45d-4ba6-b77d-504eca38c5c6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:09:02.658935 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91527e9-f45d-4ba6-b77d-504eca38c5c6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:02.660527 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91527e9-f45d-4ba6-b77d-504eca38c5c6 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:02.669081 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:02.677524 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 457/913] 10.222.0.16 () {56 vars in 1148 bytes} [Wed Aug 24 09:09:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:09:02.684197 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692780c9-03b7-4c4b-8b0d-290456de4416 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:02.792463 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692780c9-03b7-4c4b-8b0d-290456de4416 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:02.825568 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:02.833410 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d9829a-0d6f-4650-8e12-1fc4200ac9bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:02.834026 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d9829a-0d6f-4650-8e12-1fc4200ac9bd None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:02.834306 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d9829a-0d6f-4650-8e12-1fc4200ac9bd None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:02.834576 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d9829a-0d6f-4650-8e12-1fc4200ac9bd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:02.835287 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d9829a-0d6f-4650-8e12-1fc4200ac9bd None admin] RBAC: Adding query filter params (role.id=716e298774d64bc89a265236b7b9db7f, scope.project.id=c467e88e2fb0491bacefcf6d2ce326d9, user.id=0489462287fc4aea8e8e720af24e1ef8) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:09:02.835606 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d9829a-0d6f-4650-8e12-1fc4200ac9bd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:02.837024 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d9829a-0d6f-4650-8e12-1fc4200ac9bd None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:02.842703 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 457/914] 10.222.0.16 () {56 vars in 1374 bytes} [Wed Aug 24 09:09:02 2022] GET /identity/v3/role_assignments?user.id=0489462287fc4aea8e8e720af24e1ef8&scope.project.id=c467e88e2fb0491bacefcf6d2ce326d9&role.id=716e298774d64bc89a265236b7b9db7f => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:09:02.845962 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91527e9-f45d-4ba6-b77d-504eca38c5c6 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:02.971886 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91527e9-f45d-4ba6-b77d-504eca38c5c6 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:03.015941 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b453c0f-a566-401a-a4b2-c4fde3dbc2cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:03.015941 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b453c0f-a566-401a-a4b2-c4fde3dbc2cb None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:03.016220 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b453c0f-a566-401a-a4b2-c4fde3dbc2cb None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:03.016495 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b453c0f-a566-401a-a4b2-c4fde3dbc2cb None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:03.017073 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b453c0f-a566-401a-a4b2-c4fde3dbc2cb None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:03.018423 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b453c0f-a566-401a-a4b2-c4fde3dbc2cb None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:03.028531 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-7b453c0f-a566-401a-a4b2-c4fde3dbc2cb None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 09:09:03.028531 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:09:03.028531 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 09:09:03.028531 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 09:09:03.028531 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 09:09:03.028531 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 09:09:03.028531 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 09:09:03.028531 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:09:03.028531 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 09:09:03.028531 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 24 09:09:03.028531 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 09:09:03.028531 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 24 09:09:03.028531 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:09:03.028531 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:09:03.028531 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:09:03.028531 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:09:03.028531 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:09:03.029582 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:09:03.029582 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:09:03.029582 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:09:03.029582 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:09:03.029582 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:09:03.029582 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:09:03.029582 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 09:09:03.029582 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 09:09:03.029582 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 09:09:03.029582 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 09:09:03.029582 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:09:03.029582 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:09:03.029582 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:09:03.029582 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:09:03.029582 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:09:03.029582 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:09:03.030428 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:09:03.030428 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 24 09:09:03.030428 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:09:03.030428 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:09:03.030428 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:09:03.030428 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:09:03.030428 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:09:03.030428 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:09:03.030428 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:09:03.030428 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:09:03.030428 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 09:09:03.030428 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:09:03.030428 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 09:09:03.030428 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 24 09:09:03.030428 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 458/915] 10.222.0.16 () {56 vars in 1114 bytes} [Wed Aug 24 09:09:02 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:09:03.037744 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d9829a-0d6f-4650-8e12-1fc4200ac9bd None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:03.147958 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d9829a-0d6f-4650-8e12-1fc4200ac9bd None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:03.186359 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b8bb0c-d881-4a81-8878-04f4943a0b67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:03.188110 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b8bb0c-d881-4a81-8878-04f4943a0b67 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:03.188110 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b8bb0c-d881-4a81-8878-04f4943a0b67 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:03.188110 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b8bb0c-d881-4a81-8878-04f4943a0b67 None admin] PATH_INFO: `/v3/services` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:03.188110 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b8bb0c-d881-4a81-8878-04f4943a0b67 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:09:03.188491 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b8bb0c-d881-4a81-8878-04f4943a0b67 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:03.189823 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b8bb0c-d881-4a81-8878-04f4943a0b67 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:03.195147 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 458/916] 10.222.0.16 () {56 vars in 1114 bytes} [Wed Aug 24 09:09:03 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:09:03.201095 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b453c0f-a566-401a-a4b2-c4fde3dbc2cb None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:03.269704 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:03.311513 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b453c0f-a566-401a-a4b2-c4fde3dbc2cb None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:03.349244 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8c8f97-fc35-437a-9e2b-1da2802d48cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:03.349861 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8c8f97-fc35-437a-9e2b-1da2802d48cd None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:03.350192 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8c8f97-fc35-437a-9e2b-1da2802d48cd None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:03.350473 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8c8f97-fc35-437a-9e2b-1da2802d48cd None admin] PATH_INFO: `/v3/services` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:03.351115 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8c8f97-fc35-437a-9e2b-1da2802d48cd None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:09:03.351417 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8c8f97-fc35-437a-9e2b-1da2802d48cd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:03.353463 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8c8f97-fc35-437a-9e2b-1da2802d48cd None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:03.359102 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 459/917] 10.222.0.16 () {56 vars in 1114 bytes} [Wed Aug 24 09:09:03 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:09:03.364860 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b8bb0c-d881-4a81-8878-04f4943a0b67 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:03.424197 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:03.472156 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b8bb0c-d881-4a81-8878-04f4943a0b67 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:03.510954 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8b9d3f-d9b5-4a43-96dd-9b0125688084 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:03.511574 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8b9d3f-d9b5-4a43-96dd-9b0125688084 None admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:03.511928 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8b9d3f-d9b5-4a43-96dd-9b0125688084 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:03.512209 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8b9d3f-d9b5-4a43-96dd-9b0125688084 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:03.512884 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8b9d3f-d9b5-4a43-96dd-9b0125688084 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:03.514271 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8b9d3f-d9b5-4a43-96dd-9b0125688084 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:03.590912 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 459/918] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 24 09:09:03 2022] POST /identity/v3/endpoints => generated 346 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:09:03.621957 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d3720c-8f8d-496d-8d34-59367a283b74 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:03.622405 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d3720c-8f8d-496d-8d34-59367a283b74 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:03.622680 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d3720c-8f8d-496d-8d34-59367a283b74 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:03.624119 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 460/919] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:09:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:09:03.629959 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ba0b6d-ac41-44dd-8c5c-22003e84a436 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:03.630380 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ba0b6d-ac41-44dd-8c5c-22003e84a436 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:03.630657 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ba0b6d-ac41-44dd-8c5c-22003e84a436 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:03.651806 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-41ba0b6d-ac41-44dd-8c5c-22003e84a436 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:09:03.653084 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-41ba0b6d-ac41-44dd-8c5c-22003e84a436 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:09:03.698327 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc484d4-ecaa-4055-98c1-388c4f1f6498 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:03.698646 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc484d4-ecaa-4055-98c1-388c4f1f6498 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:03.698930 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc484d4-ecaa-4055-98c1-388c4f1f6498 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:03.700279 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 461/920] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:09:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:09:03.707906 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e2684d-64df-4570-a4bd-969716345642 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:03.708203 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e2684d-64df-4570-a4bd-969716345642 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:03.708450 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e2684d-64df-4570-a4bd-969716345642 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:03.728201 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-02e2684d-64df-4570-a4bd-969716345642 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:09:03.729438 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-02e2684d-64df-4570-a4bd-969716345642 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:09:03.734849 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 460/921] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:09:03 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 09:09:03.804813 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 462/922] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:09:03 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 09:09:04.007317 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ba0b6d-ac41-44dd-8c5c-22003e84a436 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:04.056781 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:04.084789 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e2684d-64df-4570-a4bd-969716345642 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:04.127237 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ba0b6d-ac41-44dd-8c5c-22003e84a436 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:04.152365 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:04.190982 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a364dd55-051e-4474-9488-0e4063d7e731 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:04.191652 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a364dd55-051e-4474-9488-0e4063d7e731 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:04.191933 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a364dd55-051e-4474-9488-0e4063d7e731 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:04.192207 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a364dd55-051e-4474-9488-0e4063d7e731 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:04.192787 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a364dd55-051e-4474-9488-0e4063d7e731 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:04.194148 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a364dd55-051e-4474-9488-0e4063d7e731 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:04.203128 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e2684d-64df-4570-a4bd-969716345642 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:04.211574 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-a364dd55-051e-4474-9488-0e4063d7e731 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 09:09:04.211574 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:09:04.211574 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 09:09:04.211574 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 09:09:04.211574 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 09:09:04.211574 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 09:09:04.211574 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 09:09:04.211574 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:09:04.211574 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 09:09:04.211574 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Aug 24 09:09:04.211574 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 09:09:04.211574 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Aug 24 09:09:04.211574 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:09:04.211574 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:09:04.211574 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:09:04.211574 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:09:04.211574 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:09:04.218634 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:09:04.218634 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:09:04.218634 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:09:04.218634 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:09:04.218634 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:09:04.218634 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:09:04.218634 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 09:09:04.218634 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 09:09:04.218634 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 09:09:04.218634 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 09:09:04.218634 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:09:04.218634 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:09:04.218634 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:09:04.218634 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:09:04.218634 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:09:04.218634 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:09:04.219482 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:09:04.219482 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Aug 24 09:09:04.219482 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:09:04.219482 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:09:04.219482 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:09:04.219482 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:09:04.219482 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:09:04.219482 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:09:04.219482 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:09:04.219482 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:09:04.219482 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 09:09:04.219482 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:09:04.219482 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 09:09:04.219482 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Aug 24 09:09:04.219482 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 461/923] 10.222.0.16 () {56 vars in 1126 bytes} [Wed Aug 24 09:09:04 2022] GET /identity/v3/services/object-store => generated 93 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:09:04.220327 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a364dd55-051e-4474-9488-0e4063d7e731 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:04.251614 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140303ee-12d8-451a-bc93-ee129b0b08ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:04.251614 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140303ee-12d8-451a-bc93-ee129b0b08ef None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:04.251614 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140303ee-12d8-451a-bc93-ee129b0b08ef None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:04.251614 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140303ee-12d8-451a-bc93-ee129b0b08ef None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:04.252401 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140303ee-12d8-451a-bc93-ee129b0b08ef None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:04.255602 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140303ee-12d8-451a-bc93-ee129b0b08ef None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:04.273979 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-140303ee-12d8-451a-bc93-ee129b0b08ef None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 09:09:04.273979 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:09:04.273979 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 09:09:04.273979 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 09:09:04.273979 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 09:09:04.273979 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 09:09:04.273979 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 09:09:04.273979 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:09:04.273979 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 09:09:04.273979 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 24 09:09:04.273979 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 09:09:04.273979 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 24 09:09:04.273979 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:09:04.273979 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:09:04.273979 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:09:04.273979 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:09:04.276217 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:09:04.276217 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:09:04.276217 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:09:04.276217 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:09:04.276217 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:09:04.276217 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:09:04.276217 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:09:04.276217 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 09:09:04.276217 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 09:09:04.276217 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 09:09:04.276217 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 09:09:04.276217 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:09:04.276217 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:09:04.276217 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:09:04.276217 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:09:04.276217 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:09:04.276217 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:09:04.278444 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:09:04.278444 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 24 09:09:04.278444 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:09:04.278444 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:09:04.278444 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:09:04.278444 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:09:04.278444 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:09:04.278444 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:09:04.278444 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:09:04.278444 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:09:04.278444 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 09:09:04.278444 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:09:04.278444 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 09:09:04.278444 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 24 09:09:04.278444 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 463/924] 10.222.0.16 () {56 vars in 1132 bytes} [Wed Aug 24 09:09:04 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 09:09:04.291641 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140303ee-12d8-451a-bc93-ee129b0b08ef None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:04.439616 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a364dd55-051e-4474-9488-0e4063d7e731 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:04.443017 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:04.496679 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140303ee-12d8-451a-bc93-ee129b0b08ef None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:04.527755 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a9a500-d577-468a-ac7b-e46bd9af8051 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:04.529020 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a9a500-d577-468a-ac7b-e46bd9af8051 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:04.529497 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a9a500-d577-468a-ac7b-e46bd9af8051 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:04.529893 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:04.530199 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a9a500-d577-468a-ac7b-e46bd9af8051 None admin] PATH_INFO: `/v3/services` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:04.531164 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a9a500-d577-468a-ac7b-e46bd9af8051 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:09:04.531785 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a9a500-d577-468a-ac7b-e46bd9af8051 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:04.533362 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a9a500-d577-468a-ac7b-e46bd9af8051 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:04.538838 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 462/925] 10.222.0.16 () {56 vars in 1122 bytes} [Wed Aug 24 09:09:04 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:09:04.549496 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a9a500-d577-468a-ac7b-e46bd9af8051 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:04.556903 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb672de8-6659-479f-bad8-614e9e1d94fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:04.557537 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb672de8-6659-479f-bad8-614e9e1d94fe None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:04.557936 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb672de8-6659-479f-bad8-614e9e1d94fe None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:04.558106 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb672de8-6659-479f-bad8-614e9e1d94fe None admin] PATH_INFO: `/v3/services` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:04.558724 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb672de8-6659-479f-bad8-614e9e1d94fe None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:09:04.559200 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb672de8-6659-479f-bad8-614e9e1d94fe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:04.563188 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb672de8-6659-479f-bad8-614e9e1d94fe None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:04.568800 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 464/926] 10.222.0.16 () {56 vars in 1126 bytes} [Wed Aug 24 09:09:04 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:09:04.575165 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb672de8-6659-479f-bad8-614e9e1d94fe None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:04.689566 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb672de8-6659-479f-bad8-614e9e1d94fe None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:04.698189 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a9a500-d577-468a-ac7b-e46bd9af8051 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:04.737476 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14755fef-d58c-4447-bcb0-3fff31a1e44e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:04.738343 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14755fef-d58c-4447-bcb0-3fff31a1e44e None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:04.738784 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14755fef-d58c-4447-bcb0-3fff31a1e44e None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:04.739212 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14755fef-d58c-4447-bcb0-3fff31a1e44e None admin] PATH_INFO: `/v3/services` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:04.740013 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14755fef-d58c-4447-bcb0-3fff31a1e44e None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:09:04.740455 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14755fef-d58c-4447-bcb0-3fff31a1e44e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:04.741961 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14755fef-d58c-4447-bcb0-3fff31a1e44e None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:04.752251 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef751a3f-3b90-43a5-99d1-6efa2f645e63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:04.753429 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef751a3f-3b90-43a5-99d1-6efa2f645e63 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:04.753641 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef751a3f-3b90-43a5-99d1-6efa2f645e63 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:04.753854 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 465/927] 10.222.0.16 () {56 vars in 1126 bytes} [Wed Aug 24 09:09:04 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:09:04.754114 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef751a3f-3b90-43a5-99d1-6efa2f645e63 None admin] PATH_INFO: `/v3/services` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:04.761873 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef751a3f-3b90-43a5-99d1-6efa2f645e63 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:09:04.761873 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef751a3f-3b90-43a5-99d1-6efa2f645e63 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:04.761873 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef751a3f-3b90-43a5-99d1-6efa2f645e63 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:04.762179 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14755fef-d58c-4447-bcb0-3fff31a1e44e None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:04.769472 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 463/928] 10.222.0.16 () {56 vars in 1122 bytes} [Wed Aug 24 09:09:04 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:09:04.770559 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef751a3f-3b90-43a5-99d1-6efa2f645e63 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:04.805001 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:04.815169 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:04.869147 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14755fef-d58c-4447-bcb0-3fff31a1e44e None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:04.880891 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef751a3f-3b90-43a5-99d1-6efa2f645e63 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:04.906127 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c0851e-c139-4d5e-a4c2-4632169213ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:04.907000 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c0851e-c139-4d5e-a4c2-4632169213ff None admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:04.907446 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c0851e-c139-4d5e-a4c2-4632169213ff None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:04.907885 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c0851e-c139-4d5e-a4c2-4632169213ff None admin] PATH_INFO: `/v3/endpoints` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:04.908702 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c0851e-c139-4d5e-a4c2-4632169213ff None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:04.910224 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c0851e-c139-4d5e-a4c2-4632169213ff None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:04.919219 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddccc19-acfa-4bd4-8950-f3bda3b2757b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:04.919873 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddccc19-acfa-4bd4-8950-f3bda3b2757b None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:04.920386 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddccc19-acfa-4bd4-8950-f3bda3b2757b None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:04.920741 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddccc19-acfa-4bd4-8950-f3bda3b2757b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:04.921461 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddccc19-acfa-4bd4-8950-f3bda3b2757b None admin] RBAC: Adding query filter params (interface=public, service_id=f0445fbd21444c45a7931845d16b4809, region_id=RegionOne) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:09:04.921773 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddccc19-acfa-4bd4-8950-f3bda3b2757b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:04.923869 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddccc19-acfa-4bd4-8950-f3bda3b2757b None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:04.929996 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 464/929] 10.222.0.16 () {56 vars in 1251 bytes} [Wed Aug 24 09:09:04 2022] GET /identity/v3/endpoints?service_id=f0445fbd21444c45a7931845d16b4809&interface=public®ion_id=RegionOne => generated 194 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:09:04.935404 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddccc19-acfa-4bd4-8950-f3bda3b2757b None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:05.010181 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 466/930] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 24 09:09:04 2022] POST /identity/v3/endpoints => generated 347 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:09:05.011930 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bcd479-2793-4c3b-afa9-550422c1e425 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:05.012308 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bcd479-2793-4c3b-afa9-550422c1e425 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:05.012639 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bcd479-2793-4c3b-afa9-550422c1e425 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:05.014112 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 467/931] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:09:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:09:05.019905 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572e68e0-4a09-43e7-8489-358a7f7c8368 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:05.020243 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572e68e0-4a09-43e7-8489-358a7f7c8368 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:05.020590 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572e68e0-4a09-43e7-8489-358a7f7c8368 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:05.041013 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:05.042207 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-572e68e0-4a09-43e7-8489-358a7f7c8368 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:09:05.043507 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-572e68e0-4a09-43e7-8489-358a7f7c8368 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:09:05.052020 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddccc19-acfa-4bd4-8950-f3bda3b2757b None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:05.098305 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5b8171-5f0d-4b97-b33c-24acca7776ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:05.098922 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5b8171-5f0d-4b97-b33c-24acca7776ec None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:05.099267 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5b8171-5f0d-4b97-b33c-24acca7776ec None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:05.099677 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5b8171-5f0d-4b97-b33c-24acca7776ec None admin] PATH_INFO: `/v3/services` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:05.100423 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5b8171-5f0d-4b97-b33c-24acca7776ec None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:05.101859 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5b8171-5f0d-4b97-b33c-24acca7776ec None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:05.115627 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 465/932] 10.222.0.16 () {56 vars in 1087 bytes} [Wed Aug 24 09:09:04 2022] GET /identity/v3/services => generated 1497 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:09:05.147796 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 468/933] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:09:05 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 09:09:05.425024 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5b8171-5f0d-4b97-b33c-24acca7776ec None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:05.535890 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5b8171-5f0d-4b97-b33c-24acca7776ec None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:05.573394 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecb5f4a-4db3-4bbb-a8c0-5d943f2cf9dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:05.574402 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecb5f4a-4db3-4bbb-a8c0-5d943f2cf9dd None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:05.574892 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecb5f4a-4db3-4bbb-a8c0-5d943f2cf9dd None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:05.575373 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecb5f4a-4db3-4bbb-a8c0-5d943f2cf9dd None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:05.589156 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecb5f4a-4db3-4bbb-a8c0-5d943f2cf9dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:09:05.594726 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-8ecb5f4a-4db3-4bbb-a8c0-5d943f2cf9dd None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 09:09:05.594726 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:09:05.594726 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:09:05.594726 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:09:05.594726 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:09:05.594726 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:09:05.594726 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:09:05.594726 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:09:05.594726 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:09:05.594726 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:09:05.594726 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:09:05.594726 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:09:05.594726 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:09:05.594726 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:09:05.594726 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:09:05.594726 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:09:05.595683 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:09:05.595683 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:09:05.595683 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:09:05.595683 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:09:05.595683 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:09:05.595683 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:09:05.595683 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:09:05.595683 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:09:05.595683 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:09:05.595683 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:09:05.595683 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:09:05.595683 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Aug 24 09:09:05.595683 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:09:05.595683 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:09:05.595683 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:09:05.595683 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:09:05.596543 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:09:05.596543 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:09:05.596543 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:09:05.596543 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:09:05.596543 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:09:05.596543 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:09:05.596543 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:09:05.596543 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:09:05.596543 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:09:05.596543 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:09:05.596543 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 09:09:05.596543 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Aug 24 09:09:05.599115 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 466/934] 10.222.0.16 () {56 vars in 1120 bytes} [Wed Aug 24 09:09:05 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:09:05.605376 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572e68e0-4a09-43e7-8489-358a7f7c8368 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:05.640287 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:05.711517 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572e68e0-4a09-43e7-8489-358a7f7c8368 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:05.730468 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37419c0-b055-471b-bb02-f8fb0185ad13 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:05.730776 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37419c0-b055-471b-bb02-f8fb0185ad13 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:05.731029 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37419c0-b055-471b-bb02-f8fb0185ad13 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:05.732694 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 467/935] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:09:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:09:05.737780 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ab6450-ab00-45b1-9ed3-e9b01356a10a None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:05.738068 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ab6450-ab00-45b1-9ed3-e9b01356a10a None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:05.738343 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ab6450-ab00-45b1-9ed3-e9b01356a10a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:05.748995 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1c03d0-5fca-4f81-8280-a2b2389aa2bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:05.749664 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1c03d0-5fca-4f81-8280-a2b2389aa2bf None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:05.749938 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1c03d0-5fca-4f81-8280-a2b2389aa2bf None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:05.750204 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1c03d0-5fca-4f81-8280-a2b2389aa2bf None admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:05.750871 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1c03d0-5fca-4f81-8280-a2b2389aa2bf None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:09:05.751711 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1c03d0-5fca-4f81-8280-a2b2389aa2bf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:05.753529 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1c03d0-5fca-4f81-8280-a2b2389aa2bf None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:05.757070 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-45ab6450-ab00-45b1-9ed3-e9b01356a10a None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:09:05.758274 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-45ab6450-ab00-45b1-9ed3-e9b01356a10a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:09:05.767679 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 469/936] 10.222.0.16 () {56 vars in 1115 bytes} [Wed Aug 24 09:09:05 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:09:05.773650 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:05.774108 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1c03d0-5fca-4f81-8280-a2b2389aa2bf None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:05.847731 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 468/937] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:09:05 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 09:09:05.894796 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1c03d0-5fca-4f81-8280-a2b2389aa2bf None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:05.904106 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:05.929431 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca859a9-2336-41a5-a8a2-7efacf4ced50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:05.930069 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca859a9-2336-41a5-a8a2-7efacf4ced50 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:05.930351 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca859a9-2336-41a5-a8a2-7efacf4ced50 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:05.930715 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca859a9-2336-41a5-a8a2-7efacf4ced50 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:05.942449 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca859a9-2336-41a5-a8a2-7efacf4ced50 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:05.943933 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca859a9-2336-41a5-a8a2-7efacf4ced50 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:05.958433 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-2ca859a9-2336-41a5-a8a2-7efacf4ced50 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:09:05.958433 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:09:05.958433 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:09:05.958433 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:09:05.958433 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:09:05.958433 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:09:05.958433 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:09:05.958433 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:09:05.958433 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:09:05.958433 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:09:05.958433 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:09:05.958433 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:09:05.958433 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:09:05.958433 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:09:05.958433 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:09:05.958433 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:09:05.959373 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:09:05.959373 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:09:05.959373 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:09:05.959373 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:09:05.959373 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:09:05.959373 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:09:05.959373 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:09:05.959373 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 24 09:09:05.959373 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:09:05.959373 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:09:05.959373 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:09:05.959373 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:09:05.959373 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:09:05.959373 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:09:05.959373 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:09:05.959373 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:09:05.959373 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:09:05.960298 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:09:05.960298 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:09:05.960298 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 24 09:09:05.963979 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 470/938] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 24 09:09:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:09:05.965620 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ab6450-ab00-45b1-9ed3-e9b01356a10a None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:06.071329 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ab6450-ab00-45b1-9ed3-e9b01356a10a None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:06.107610 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f68e73b-ac84-476c-ab07-b56d54851267 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:06.108237 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f68e73b-ac84-476c-ab07-b56d54851267 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:06.108522 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f68e73b-ac84-476c-ab07-b56d54851267 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:06.108796 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f68e73b-ac84-476c-ab07-b56d54851267 None admin] PATH_INFO: `/v3/domains` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:06.109446 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f68e73b-ac84-476c-ab07-b56d54851267 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:09:06.109751 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f68e73b-ac84-476c-ab07-b56d54851267 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:06.111123 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f68e73b-ac84-476c-ab07-b56d54851267 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:06.122560 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca859a9-2336-41a5-a8a2-7efacf4ced50 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:06.126359 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 469/939] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Aug 24 09:09:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:09:06.136827 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f68e73b-ac84-476c-ab07-b56d54851267 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:06.141680 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:06.246038 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f68e73b-ac84-476c-ab07-b56d54851267 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:06.246484 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca859a9-2336-41a5-a8a2-7efacf4ced50 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:06.281984 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af45c98a-f709-44b2-8c9b-c150878c807d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:06.282621 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af45c98a-f709-44b2-8c9b-c150878c807d None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:06.282905 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af45c98a-f709-44b2-8c9b-c150878c807d None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:06.283180 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af45c98a-f709-44b2-8c9b-c150878c807d None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:06.286357 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03d9e59-5bb4-4552-be5c-f302c688bf30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:06.287194 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03d9e59-5bb4-4552-be5c-f302c688bf30 None admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:06.287776 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03d9e59-5bb4-4552-be5c-f302c688bf30 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:06.288256 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03d9e59-5bb4-4552-be5c-f302c688bf30 None admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:06.289116 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03d9e59-5bb4-4552-be5c-f302c688bf30 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:06.291343 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03d9e59-5bb4-4552-be5c-f302c688bf30 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:06.291937 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af45c98a-f709-44b2-8c9b-c150878c807d None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:06.293369 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af45c98a-f709-44b2-8c9b-c150878c807d None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:06.296280 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: WARNING keystone.server.flask.common [None req-a03d9e59-5bb4-4552-be5c-f302c688bf30 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 24 09:09:06.304756 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-af45c98a-f709-44b2-8c9b-c150878c807d None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 09:09:06.304756 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:09:06.304756 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:09:06.304756 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:09:06.304756 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:09:06.304756 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:09:06.304756 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:09:06.304756 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:09:06.304756 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:09:06.304756 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:09:06.304756 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:09:06.304756 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:09:06.304756 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 09:09:06.304756 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 09:09:06.304756 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 09:09:06.304756 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 09:09:06.305708 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:09:06.305708 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:09:06.305708 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 09:09:06.305708 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:09:06.305708 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 09:09:06.305708 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:09:06.305708 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:09:06.305708 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:09:06.305708 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:09:06.305708 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:09:06.305708 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:09:06.305708 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Aug 24 09:09:06.305708 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:09:06.305708 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:09:06.305708 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:09:06.305708 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:09:06.305708 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:09:06.305708 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:09:06.306652 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:09:06.306652 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:09:06.306652 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 09:09:06.306652 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 09:09:06.306652 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 09:09:06.306652 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 09:09:06.306652 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 09:09:06.306652 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 09:09:06.306652 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 09:09:06.306652 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Aug 24 09:09:06.310678 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 470/940] 10.222.0.16 () {56 vars in 1117 bytes} [Wed Aug 24 09:09:06 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 09:09:06.311961 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af45c98a-f709-44b2-8c9b-c150878c807d None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:06.357816 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.sql.core [None req-a03d9e59-5bb4-4552-be5c-f302c688bf30 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 24 09:09:06.357816 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 24 09:09:06.357816 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [parameters: {'id': 'a94c81587ba4479dbce80ec5c34c01eb', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 24 09:09:06.357816 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81602) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 24 09:09:06.365891 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-a03d9e59-5bb4-4552-be5c-f302c688bf30 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id a94c81587ba4479dbce80ec5c34c01eb.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id a94c81587ba4479dbce80ec5c34c01eb. Aug 24 09:09:06.365891 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:09:06.365891 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 09:09:06.365891 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 09:09:06.365891 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 09:09:06.365891 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 09:09:06.365891 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 09:09:06.365891 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 09:09:06.365891 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 09:09:06.365891 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application conn.query(q) Aug 24 09:09:06.365891 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 09:09:06.365891 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 09:09:06.365891 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 09:09:06.366700 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application result.read() Aug 24 09:09:06.366700 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 09:09:06.366700 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 09:09:06.366700 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 09:09:06.366700 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 09:09:06.366700 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 09:09:06.366700 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 09:09:06.366700 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 09:09:06.366700 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 09:09:06.366700 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 24 09:09:06.366700 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 24 09:09:06.366700 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 24 09:09:06.366700 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 24 09:09:06.366700 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:09:06.366700 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 24 09:09:06.366700 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 24 09:09:06.366700 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Aug 24 09:09:06.366700 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return project_ref.to_dict() Aug 24 09:09:06.367693 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 09:09:06.367693 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application next(self.gen) Aug 24 09:09:06.367693 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 24 09:09:06.367693 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application yield resource Aug 24 09:09:06.367693 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 09:09:06.367693 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application next(self.gen) Aug 24 09:09:06.367693 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 24 09:09:06.367693 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application self.session.rollback() Aug 24 09:09:06.367693 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 24 09:09:06.367693 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application self.force_reraise() Aug 24 09:09:06.367693 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 24 09:09:06.367693 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise self.value Aug 24 09:09:06.367693 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 24 09:09:06.367693 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 24 09:09:06.367693 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 24 09:09:06.367693 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application session.commit() Aug 24 09:09:06.367693 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 24 09:09:06.368633 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 24 09:09:06.368633 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 24 09:09:06.368633 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application self._prepare_impl() Aug 24 09:09:06.368633 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 24 09:09:06.368633 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application self.session.flush() Aug 24 09:09:06.368633 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 24 09:09:06.368633 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application self._flush(objects) Aug 24 09:09:06.368633 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 24 09:09:06.368633 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 24 09:09:06.368633 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 24 09:09:06.368633 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application compat.raise_( Aug 24 09:09:06.368633 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 09:09:06.368633 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception Aug 24 09:09:06.368633 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 24 09:09:06.368633 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application flush_context.execute() Aug 24 09:09:06.368633 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 24 09:09:06.368633 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rec.execute(self) Aug 24 09:09:06.369552 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 24 09:09:06.369552 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 24 09:09:06.369552 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 24 09:09:06.369552 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 24 09:09:06.369552 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 24 09:09:06.369552 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 24 09:09:06.369552 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 24 09:09:06.369552 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 24 09:09:06.369552 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 24 09:09:06.369552 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 24 09:09:06.369552 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 24 09:09:06.369552 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 24 09:09:06.369552 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 24 09:09:06.369552 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 24 09:09:06.369552 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 24 09:09:06.370364 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 24 09:09:06.370364 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 09:09:06.370364 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception Aug 24 09:09:06.370364 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 09:09:06.370364 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 09:09:06.370364 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 09:09:06.370364 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 09:09:06.370364 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 09:09:06.370364 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 09:09:06.370364 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 09:09:06.370364 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application conn.query(q) Aug 24 09:09:06.370364 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 09:09:06.370364 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 09:09:06.370364 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 09:09:06.370364 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application result.read() Aug 24 09:09:06.370364 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 09:09:06.370364 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 09:09:06.371249 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 09:09:06.371249 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 09:09:06.371249 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 09:09:06.371249 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 09:09:06.371249 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 09:09:06.371249 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 09:09:06.371249 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 24 09:09:06.371249 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 24 09:09:06.371249 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [parameters: {'id': 'a94c81587ba4479dbce80ec5c34c01eb', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 24 09:09:06.371249 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 24 09:09:06.371249 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 24 09:09:06.371249 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 09:09:06.371249 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 24 09:09:06.371249 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:09:06.371249 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Aug 24 09:09:06.373792 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Aug 24 09:09:06.373792 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 24 09:09:06.373792 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 24 09:09:06.373792 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Aug 24 09:09:06.373792 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 24 09:09:06.373792 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 09:09:06.373792 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 24 09:09:06.373792 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:09:06.373792 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:09:06.373792 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:09:06.373792 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:09:06.373792 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:09:06.373792 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:09:06.373792 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:09:06.373792 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:09:06.373792 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:09:06.373792 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:09:06.375962 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:09:06.375962 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Aug 24 09:09:06.375962 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Aug 24 09:09:06.375962 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:09:06.375962 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:09:06.375962 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Aug 24 09:09:06.375962 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.Conflict( Aug 24 09:09:06.375962 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id a94c81587ba4479dbce80ec5c34c01eb. Aug 24 09:09:06.375962 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 24 09:09:06.378832 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:06.384923 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 471/941] 10.222.0.16 () {60 vars in 1140 bytes} [Wed Aug 24 09:09:06 2022] POST /identity/v3/projects => generated 262 bytes in 257 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 24 09:09:06.386420 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03d9e59-5bb4-4552-be5c-f302c688bf30 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:06.390792 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:06.424760 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af45c98a-f709-44b2-8c9b-c150878c807d None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:06.462721 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac668fc-ccd1-4776-857b-fcca6a12d480 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:06.463273 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac668fc-ccd1-4776-857b-fcca6a12d480 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:06.463684 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac668fc-ccd1-4776-857b-fcca6a12d480 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:06.463962 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac668fc-ccd1-4776-857b-fcca6a12d480 None admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:06.464702 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac668fc-ccd1-4776-857b-fcca6a12d480 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:09:06.465011 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac668fc-ccd1-4776-857b-fcca6a12d480 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:06.466697 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac668fc-ccd1-4776-857b-fcca6a12d480 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:06.490507 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 471/942] 10.222.0.16 () {56 vars in 1149 bytes} [Wed Aug 24 09:09:06 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:09:06.491979 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac668fc-ccd1-4776-857b-fcca6a12d480 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:06.500435 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03d9e59-5bb4-4552-be5c-f302c688bf30 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:06.538587 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6d0acd-bb63-4040-8240-abbaf2d57025 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:06.539541 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6d0acd-bb63-4040-8240-abbaf2d57025 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:06.540693 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6d0acd-bb63-4040-8240-abbaf2d57025 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:06.541140 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6d0acd-bb63-4040-8240-abbaf2d57025 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:06.548739 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6d0acd-bb63-4040-8240-abbaf2d57025 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:06.550630 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6d0acd-bb63-4040-8240-abbaf2d57025 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:06.563177 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-6c6d0acd-bb63-4040-8240-abbaf2d57025 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:09:06.563177 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:09:06.563177 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:09:06.563177 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:09:06.563177 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:09:06.563177 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:09:06.563177 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:09:06.563177 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:09:06.563177 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:09:06.563177 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:09:06.563177 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:09:06.563177 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:09:06.563177 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 09:09:06.563177 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 09:09:06.563177 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 09:09:06.564181 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 09:09:06.564181 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:09:06.564181 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:09:06.564181 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:09:06.564181 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:09:06.564181 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:09:06.564181 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:09:06.564181 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:09:06.564181 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 24 09:09:06.564181 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:09:06.564181 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:09:06.564181 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:09:06.564181 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:09:06.564181 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:09:06.564181 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:09:06.564181 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:09:06.564181 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:09:06.565068 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 09:09:06.565068 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 09:09:06.565068 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 09:09:06.565068 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 09:09:06.565068 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 09:09:06.565068 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 09:09:06.565068 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:09:06.565068 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 24 09:09:06.572368 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 472/943] 10.222.0.16 () {56 vars in 1111 bytes} [Wed Aug 24 09:09:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:09:06.573667 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6d0acd-bb63-4040-8240-abbaf2d57025 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:06.599841 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac668fc-ccd1-4776-857b-fcca6a12d480 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:06.624623 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:06.637076 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596cafda-cd81-4e32-9129-cdd90dd3e2cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:06.637722 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596cafda-cd81-4e32-9129-cdd90dd3e2cd None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:06.638010 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596cafda-cd81-4e32-9129-cdd90dd3e2cd None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:06.643111 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596cafda-cd81-4e32-9129-cdd90dd3e2cd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:06.645982 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:06.649351 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596cafda-cd81-4e32-9129-cdd90dd3e2cd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:06.651562 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596cafda-cd81-4e32-9129-cdd90dd3e2cd None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:06.667454 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-596cafda-cd81-4e32-9129-cdd90dd3e2cd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:09:06.667454 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:09:06.667454 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:09:06.667454 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:09:06.667454 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:09:06.667454 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:09:06.667454 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:09:06.667454 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:09:06.667454 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:09:06.667454 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:09:06.667454 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:09:06.667454 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:09:06.667454 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:09:06.667454 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:09:06.667454 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:09:06.667454 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:09:06.674748 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:09:06.674748 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:09:06.674748 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:09:06.674748 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:09:06.674748 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:09:06.674748 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:09:06.674748 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:09:06.674748 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Aug 24 09:09:06.674748 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:09:06.674748 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:09:06.674748 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:09:06.674748 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:09:06.674748 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:09:06.674748 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:09:06.674748 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:09:06.674748 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:09:06.674748 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:09:06.675695 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:09:06.675695 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:09:06.675695 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Aug 24 09:09:06.675695 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 472/944] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 24 09:09:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:09:06.675695 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596cafda-cd81-4e32-9129-cdd90dd3e2cd None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:06.691250 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6d0acd-bb63-4040-8240-abbaf2d57025 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:06.729559 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e08bc5f-5883-42c7-93df-725c9dcedeb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:06.730511 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e08bc5f-5883-42c7-93df-725c9dcedeb7 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:06.731108 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e08bc5f-5883-42c7-93df-725c9dcedeb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:06.731738 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e08bc5f-5883-42c7-93df-725c9dcedeb7 None admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:06.732792 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e08bc5f-5883-42c7-93df-725c9dcedeb7 None admin] RBAC: Adding query filter params (name=service) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:09:06.733323 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e08bc5f-5883-42c7-93df-725c9dcedeb7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:06.735052 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e08bc5f-5883-42c7-93df-725c9dcedeb7 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:06.760009 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 473/945] 10.222.0.16 () {56 vars in 1112 bytes} [Wed Aug 24 09:09:06 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:09:06.789780 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596cafda-cd81-4e32-9129-cdd90dd3e2cd None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:06.829186 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe67839-37d4-4745-bd9b-a2330a11bd8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:06.829812 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe67839-37d4-4745-bd9b-a2330a11bd8e None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:06.830100 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe67839-37d4-4745-bd9b-a2330a11bd8e None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:06.830373 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe67839-37d4-4745-bd9b-a2330a11bd8e None admin] PATH_INFO: `/v3/domains` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:06.831032 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe67839-37d4-4745-bd9b-a2330a11bd8e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:09:06.831337 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe67839-37d4-4745-bd9b-a2330a11bd8e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:06.832957 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe67839-37d4-4745-bd9b-a2330a11bd8e None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:06.852093 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 473/946] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Aug 24 09:09:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:09:06.853899 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e08bc5f-5883-42c7-93df-725c9dcedeb7 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:06.966743 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e08bc5f-5883-42c7-93df-725c9dcedeb7 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:06.992821 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:07.004387 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3bea7a-5505-4785-8a7c-acdc339ae18a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:07.011476 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3bea7a-5505-4785-8a7c-acdc339ae18a None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:07.011476 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3bea7a-5505-4785-8a7c-acdc339ae18a None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:07.011476 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3bea7a-5505-4785-8a7c-acdc339ae18a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:07.011741 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1b4eda-fbbe-4659-9fb6-51ad4475314d None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:07.011741 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1b4eda-fbbe-4659-9fb6-51ad4475314d None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:07.011741 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1b4eda-fbbe-4659-9fb6-51ad4475314d None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:07.011741 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 474/947] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:09:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:09:07.013322 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3bea7a-5505-4785-8a7c-acdc339ae18a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:07.014958 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3bea7a-5505-4785-8a7c-acdc339ae18a None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:07.015441 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24a07ba-16f7-410e-89a5-4eb113128a96 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:07.015766 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24a07ba-16f7-410e-89a5-4eb113128a96 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:07.016049 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24a07ba-16f7-410e-89a5-4eb113128a96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:07.028381 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-fb3bea7a-5505-4785-8a7c-acdc339ae18a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:09:07.028381 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:09:07.028381 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:09:07.028381 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:09:07.028381 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:09:07.028381 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:09:07.028381 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:09:07.028381 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:09:07.028381 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:09:07.028381 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:09:07.028381 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:09:07.028381 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:09:07.028381 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 09:09:07.028381 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 09:09:07.028381 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 09:09:07.029544 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 09:09:07.029544 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:09:07.029544 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:09:07.029544 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:09:07.029544 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:09:07.029544 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:09:07.029544 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:09:07.029544 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:09:07.029544 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 24 09:09:07.029544 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:09:07.029544 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:09:07.029544 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:09:07.029544 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:09:07.029544 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:09:07.029544 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:09:07.029544 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:09:07.029544 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:09:07.030578 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 09:09:07.030578 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 09:09:07.030578 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 09:09:07.030578 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 09:09:07.030578 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 09:09:07.030578 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 09:09:07.030578 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:09:07.030578 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 24 09:09:07.030578 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 474/948] 10.222.0.16 () {56 vars in 1111 bytes} [Wed Aug 24 09:09:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:09:07.035840 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3bea7a-5505-4785-8a7c-acdc339ae18a None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:07.041827 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-a24a07ba-16f7-410e-89a5-4eb113128a96 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:09:07.043058 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-a24a07ba-16f7-410e-89a5-4eb113128a96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:09:07.074318 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:07.126067 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 475/949] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:09:07 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 09:09:07.127507 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba0aa62-e89a-493f-b882-9032bfece0d5 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:07.131889 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba0aa62-e89a-493f-b882-9032bfece0d5 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:07.132403 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba0aa62-e89a-493f-b882-9032bfece0d5 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:07.134793 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 476/950] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:09:07 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:09:07.140537 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add11ba0-7f6e-4479-becb-25d9b884370a None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:07.140840 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add11ba0-7f6e-4479-becb-25d9b884370a None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:07.141311 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add11ba0-7f6e-4479-becb-25d9b884370a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:07.160116 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-add11ba0-7f6e-4479-becb-25d9b884370a None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:09:07.161320 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-add11ba0-7f6e-4479-becb-25d9b884370a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:09:07.161825 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3bea7a-5505-4785-8a7c-acdc339ae18a None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:07.212785 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15023b1-7c0e-4589-9059-d70ac124a39f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:07.213803 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15023b1-7c0e-4589-9059-d70ac124a39f None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:07.214259 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15023b1-7c0e-4589-9059-d70ac124a39f None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:07.214869 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15023b1-7c0e-4589-9059-d70ac124a39f None admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:07.216177 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15023b1-7c0e-4589-9059-d70ac124a39f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:09:07.216664 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15023b1-7c0e-4589-9059-d70ac124a39f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:07.218266 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15023b1-7c0e-4589-9059-d70ac124a39f None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:07.234645 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 477/951] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:09:07 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 09:09:07.240179 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 475/952] 10.222.0.16 () {56 vars in 1148 bytes} [Wed Aug 24 09:09:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:09:07.246017 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add11ba0-7f6e-4479-becb-25d9b884370a None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:07.347747 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add11ba0-7f6e-4479-becb-25d9b884370a None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:07.355908 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:07.382946 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd4238a-2d2c-4a80-bce3-ac2a208439a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:07.383690 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd4238a-2d2c-4a80-bce3-ac2a208439a7 None admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:07.384535 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd4238a-2d2c-4a80-bce3-ac2a208439a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:07.384809 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd4238a-2d2c-4a80-bce3-ac2a208439a7 None admin] PATH_INFO: `/v3/projects/c467e88e2fb0491bacefcf6d2ce326d9/users/0489462287fc4aea8e8e720af24e1ef8/roles/716e298774d64bc89a265236b7b9db7f` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:07.386994 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd4238a-2d2c-4a80-bce3-ac2a208439a7 None admin] RBAC: Authorizing `identity:create_grant(project_id=c467e88e2fb0491bacefcf6d2ce326d9, user_id=0489462287fc4aea8e8e720af24e1ef8, role_id=716e298774d64bc89a265236b7b9db7f)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:07.388442 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd4238a-2d2c-4a80-bce3-ac2a208439a7 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:07.399133 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15023b1-7c0e-4589-9059-d70ac124a39f None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:07.450410 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:07.469537 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 478/953] 10.222.0.16 () {58 vars in 1439 bytes} [Wed Aug 24 09:09:07 2022] PUT /identity/v3/projects/c467e88e2fb0491bacefcf6d2ce326d9/users/0489462287fc4aea8e8e720af24e1ef8/roles/716e298774d64bc89a265236b7b9db7f => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 09:09:07.520202 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd4238a-2d2c-4a80-bce3-ac2a208439a7 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:07.524885 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15023b1-7c0e-4589-9059-d70ac124a39f None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:07.564140 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c8b2a1-f0a2-417f-8455-363dded30539 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:07.565177 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c8b2a1-f0a2-417f-8455-363dded30539 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:07.565662 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c8b2a1-f0a2-417f-8455-363dded30539 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:07.566157 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c8b2a1-f0a2-417f-8455-363dded30539 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:07.566985 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c8b2a1-f0a2-417f-8455-363dded30539 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:07.568661 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c8b2a1-f0a2-417f-8455-363dded30539 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:07.578896 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-c1c8b2a1-f0a2-417f-8455-363dded30539 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 09:09:07.578896 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:09:07.578896 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 09:09:07.578896 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 09:09:07.578896 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 09:09:07.578896 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 09:09:07.578896 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 09:09:07.578896 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:09:07.578896 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 09:09:07.578896 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 24 09:09:07.578896 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 09:09:07.578896 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 24 09:09:07.578896 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:09:07.578896 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:09:07.578896 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:09:07.578896 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:09:07.578896 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:09:07.580338 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:09:07.580338 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:09:07.580338 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:09:07.580338 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:09:07.580338 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:09:07.580338 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:09:07.580338 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 09:09:07.580338 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 09:09:07.580338 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 09:09:07.580338 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 09:09:07.580338 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:09:07.580338 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:09:07.580338 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:09:07.580338 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:09:07.580338 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:09:07.580338 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:09:07.581264 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:09:07.581264 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 24 09:09:07.581264 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:09:07.581264 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:09:07.581264 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:09:07.581264 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:09:07.581264 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:09:07.581264 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:09:07.581264 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:09:07.581264 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:09:07.581264 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 09:09:07.581264 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:09:07.581264 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 09:09:07.581264 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 24 09:09:07.584062 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 476/954] 10.222.0.16 () {56 vars in 1111 bytes} [Wed Aug 24 09:09:07 2022] GET /identity/v3/services/compute => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 09:09:07.593429 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c8b2a1-f0a2-417f-8455-363dded30539 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:07.664548 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd4238a-2d2c-4a80-bce3-ac2a208439a7 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:07.704250 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8650db-5eba-4e12-a4e7-ed76105318e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:07.704953 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8650db-5eba-4e12-a4e7-ed76105318e5 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:07.705241 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8650db-5eba-4e12-a4e7-ed76105318e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:07.705517 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8650db-5eba-4e12-a4e7-ed76105318e5 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:07.706223 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8650db-5eba-4e12-a4e7-ed76105318e5 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:07.707752 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8650db-5eba-4e12-a4e7-ed76105318e5 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:07.717598 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-0e8650db-5eba-4e12-a4e7-ed76105318e5 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 09:09:07.717598 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:09:07.717598 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 09:09:07.717598 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 09:09:07.717598 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 09:09:07.717598 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 09:09:07.717598 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 09:09:07.717598 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:09:07.717598 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 09:09:07.717598 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Aug 24 09:09:07.717598 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 09:09:07.717598 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Aug 24 09:09:07.717598 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:09:07.717598 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:09:07.717598 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:09:07.717598 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:09:07.717598 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:09:07.718627 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:09:07.718627 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:09:07.718627 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:09:07.718627 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:09:07.718627 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:09:07.718627 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:09:07.718627 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 09:09:07.718627 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 09:09:07.718627 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 09:09:07.718627 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 09:09:07.718627 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:09:07.718627 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:09:07.718627 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:09:07.718627 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:09:07.718627 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:09:07.718627 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:09:07.719475 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:09:07.719475 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Aug 24 09:09:07.719475 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:09:07.719475 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:09:07.719475 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:09:07.719475 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:09:07.719475 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:09:07.719475 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:09:07.719475 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:09:07.719475 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:09:07.719475 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 09:09:07.719475 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:09:07.719475 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 09:09:07.719475 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Aug 24 09:09:07.719475 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 479/955] 10.222.0.16 () {56 vars in 1126 bytes} [Wed Aug 24 09:09:07 2022] GET /identity/v3/services/object-store => generated 93 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:09:07.726594 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8650db-5eba-4e12-a4e7-ed76105318e5 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:07.732095 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c8b2a1-f0a2-417f-8455-363dded30539 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:07.747417 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:07.756091 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:07.772007 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52540130-ee8a-457b-b546-ba829123976c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:07.772879 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52540130-ee8a-457b-b546-ba829123976c None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:07.773432 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52540130-ee8a-457b-b546-ba829123976c None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:07.773915 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52540130-ee8a-457b-b546-ba829123976c None admin] PATH_INFO: `/v3/services` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:07.775375 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52540130-ee8a-457b-b546-ba829123976c None admin] RBAC: Adding query filter params (name=compute) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:09:07.775901 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52540130-ee8a-457b-b546-ba829123976c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:07.777507 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52540130-ee8a-457b-b546-ba829123976c None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:07.783922 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 477/956] 10.222.0.16 () {56 vars in 1112 bytes} [Wed Aug 24 09:09:07 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:09:07.789725 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52540130-ee8a-457b-b546-ba829123976c None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:07.859038 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8650db-5eba-4e12-a4e7-ed76105318e5 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:07.908594 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703cfb75-0d8b-4300-9250-e958ff74e19e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:07.909622 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703cfb75-0d8b-4300-9250-e958ff74e19e None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:07.909924 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703cfb75-0d8b-4300-9250-e958ff74e19e None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:07.910291 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703cfb75-0d8b-4300-9250-e958ff74e19e None admin] PATH_INFO: `/v3/services` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:07.910994 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703cfb75-0d8b-4300-9250-e958ff74e19e None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:09:07.911361 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703cfb75-0d8b-4300-9250-e958ff74e19e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:07.912449 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52540130-ee8a-457b-b546-ba829123976c None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:07.912884 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703cfb75-0d8b-4300-9250-e958ff74e19e None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:07.923158 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 480/957] 10.222.0.16 () {56 vars in 1122 bytes} [Wed Aug 24 09:09:07 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:09:07.925568 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703cfb75-0d8b-4300-9250-e958ff74e19e None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:07.961094 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d553e578-1912-4655-8147-e966b7eb20dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:07.962495 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d553e578-1912-4655-8147-e966b7eb20dd None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:07.962950 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d553e578-1912-4655-8147-e966b7eb20dd None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:07.963375 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d553e578-1912-4655-8147-e966b7eb20dd None admin] PATH_INFO: `/v3/services` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:07.964179 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d553e578-1912-4655-8147-e966b7eb20dd None admin] RBAC: Adding query filter params (type=compute) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:09:07.964622 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d553e578-1912-4655-8147-e966b7eb20dd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:07.966123 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d553e578-1912-4655-8147-e966b7eb20dd None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:07.971780 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 478/958] 10.222.0.16 () {56 vars in 1112 bytes} [Wed Aug 24 09:09:07 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:09:08.020007 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:08.041928 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703cfb75-0d8b-4300-9250-e958ff74e19e None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:08.082900 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181d967a-3398-47f9-a690-816783826bbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:08.083537 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181d967a-3398-47f9-a690-816783826bbe None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:08.083854 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181d967a-3398-47f9-a690-816783826bbe None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:08.084134 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181d967a-3398-47f9-a690-816783826bbe None admin] PATH_INFO: `/v3/services` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:08.084786 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181d967a-3398-47f9-a690-816783826bbe None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:09:08.085090 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181d967a-3398-47f9-a690-816783826bbe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:08.086500 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181d967a-3398-47f9-a690-816783826bbe None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:08.093221 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 481/959] 10.222.0.16 () {56 vars in 1122 bytes} [Wed Aug 24 09:09:07 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:09:08.098887 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d553e578-1912-4655-8147-e966b7eb20dd None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:08.139735 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:08.220857 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d553e578-1912-4655-8147-e966b7eb20dd None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:08.258641 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3563cd5-5862-43aa-aea5-fd026cc95462 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:08.259318 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3563cd5-5862-43aa-aea5-fd026cc95462 None admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:08.259624 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3563cd5-5862-43aa-aea5-fd026cc95462 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:08.259907 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3563cd5-5862-43aa-aea5-fd026cc95462 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:08.260584 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3563cd5-5862-43aa-aea5-fd026cc95462 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:08.261955 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3563cd5-5862-43aa-aea5-fd026cc95462 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:08.453905 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 479/960] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 24 09:09:08 2022] POST /identity/v3/endpoints => generated 349 bytes in 355 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:09:08.959032 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445b69fb-3506-4b93-a51e-2a35c9819f68 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:08.960113 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445b69fb-3506-4b93-a51e-2a35c9819f68 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:08.960696 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445b69fb-3506-4b93-a51e-2a35c9819f68 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:08.962587 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 482/961] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:09:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 09:09:08.968389 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d47a03-0e95-4983-b4cb-4f64bf2bd69d None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:08.969245 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d47a03-0e95-4983-b4cb-4f64bf2bd69d None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:08.969740 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d47a03-0e95-4983-b4cb-4f64bf2bd69d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:08.991120 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-85d47a03-0e95-4983-b4cb-4f64bf2bd69d None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:09:08.993774 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-85d47a03-0e95-4983-b4cb-4f64bf2bd69d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:09:09.075826 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 480/962] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:09:08 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 09:09:09.359106 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445b69fb-3506-4b93-a51e-2a35c9819f68 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:09.431613 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe6142a-c12c-4b2a-aa5a-527cc28b332a None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:09.431963 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe6142a-c12c-4b2a-aa5a-527cc28b332a None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:09.432186 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe6142a-c12c-4b2a-aa5a-527cc28b332a None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:09.433777 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 481/963] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:09:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 09:09:09.438690 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138869e5-d9a0-4569-af52-41a5bbf42f0d None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:09.438966 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138869e5-d9a0-4569-af52-41a5bbf42f0d None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:09.439236 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138869e5-d9a0-4569-af52-41a5bbf42f0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:09.458025 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-138869e5-d9a0-4569-af52-41a5bbf42f0d None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:09:09.459225 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-138869e5-d9a0-4569-af52-41a5bbf42f0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:09:09.475147 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445b69fb-3506-4b93-a51e-2a35c9819f68 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:09.477458 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:09.512350 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a9f7cb-a467-4f0c-b32d-30609e2c719e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:09.513056 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a9f7cb-a467-4f0c-b32d-30609e2c719e None admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:09.513336 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a9f7cb-a467-4f0c-b32d-30609e2c719e None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:09.513815 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a9f7cb-a467-4f0c-b32d-30609e2c719e None admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:09.514470 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a9f7cb-a467-4f0c-b32d-30609e2c719e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:09.515951 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a9f7cb-a467-4f0c-b32d-30609e2c719e None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:09.518550 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: WARNING keystone.server.flask.common [None req-89a9f7cb-a467-4f0c-b32d-30609e2c719e None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 24 09:09:09.549160 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 482/964] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:09:09 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 09:09:09.566670 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.sql.core [None req-89a9f7cb-a467-4f0c-b32d-30609e2c719e None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 24 09:09:09.566670 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 24 09:09:09.566670 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [parameters: {'user_id': '27d7f28a1ed2439aaf65860bcdae04f0', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 24 09:09:09.566670 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81603) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 24 09:09:09.584953 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-89a9f7cb-a467-4f0c-b32d-30609e2c719e None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 24 09:09:09.584953 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:09:09.584953 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 09:09:09.584953 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 09:09:09.584953 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 09:09:09.584953 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 09:09:09.584953 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 09:09:09.584953 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 09:09:09.584953 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 09:09:09.584953 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application conn.query(q) Aug 24 09:09:09.584953 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 09:09:09.584953 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 09:09:09.584953 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 09:09:09.584953 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application result.read() Aug 24 09:09:09.584953 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 09:09:09.585989 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 09:09:09.585989 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 09:09:09.585989 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 09:09:09.585989 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 09:09:09.585989 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 09:09:09.585989 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 09:09:09.585989 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 09:09:09.585989 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 24 09:09:09.585989 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Aug 24 09:09:09.585989 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 24 09:09:09.585989 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Aug 24 09:09:09.585989 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:09:09.585989 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 24 09:09:09.585989 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 24 09:09:09.585989 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Aug 24 09:09:09.585989 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Aug 24 09:09:09.585989 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 09:09:09.585989 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application next(self.gen) Aug 24 09:09:09.587560 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 24 09:09:09.587560 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application yield resource Aug 24 09:09:09.587560 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 09:09:09.587560 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application next(self.gen) Aug 24 09:09:09.587560 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 24 09:09:09.587560 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application self.session.rollback() Aug 24 09:09:09.587560 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 24 09:09:09.587560 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application self.force_reraise() Aug 24 09:09:09.587560 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 24 09:09:09.587560 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise self.value Aug 24 09:09:09.587560 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 24 09:09:09.587560 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 24 09:09:09.587560 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 24 09:09:09.587560 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application session.commit() Aug 24 09:09:09.587560 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 24 09:09:09.587560 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 24 09:09:09.587560 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 24 09:09:09.589268 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application self._prepare_impl() Aug 24 09:09:09.589268 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 24 09:09:09.589268 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application self.session.flush() Aug 24 09:09:09.589268 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 24 09:09:09.589268 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application self._flush(objects) Aug 24 09:09:09.589268 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 24 09:09:09.589268 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 24 09:09:09.589268 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 24 09:09:09.589268 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application compat.raise_( Aug 24 09:09:09.589268 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 09:09:09.589268 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception Aug 24 09:09:09.589268 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 24 09:09:09.589268 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application flush_context.execute() Aug 24 09:09:09.589268 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 24 09:09:09.589268 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rec.execute(self) Aug 24 09:09:09.589268 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 24 09:09:09.589268 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 24 09:09:09.590190 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 24 09:09:09.590190 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 24 09:09:09.590190 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Aug 24 09:09:09.590190 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application result = connection._execute_20( Aug 24 09:09:09.590190 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 24 09:09:09.590190 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 24 09:09:09.590190 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 24 09:09:09.590190 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 24 09:09:09.590190 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 24 09:09:09.590190 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 24 09:09:09.590190 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 24 09:09:09.590190 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 24 09:09:09.590190 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 24 09:09:09.590190 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 24 09:09:09.590190 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 09:09:09.590190 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception Aug 24 09:09:09.592125 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 09:09:09.592125 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 09:09:09.592125 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 09:09:09.592125 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 09:09:09.592125 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 09:09:09.592125 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 09:09:09.592125 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 09:09:09.592125 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application conn.query(q) Aug 24 09:09:09.592125 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 09:09:09.592125 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 09:09:09.592125 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 09:09:09.592125 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application result.read() Aug 24 09:09:09.592125 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 09:09:09.592125 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 09:09:09.592125 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 09:09:09.592125 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 09:09:09.594203 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 09:09:09.594203 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 09:09:09.594203 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 09:09:09.594203 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 09:09:09.594203 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 24 09:09:09.594203 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 24 09:09:09.594203 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [parameters: {'user_id': '27d7f28a1ed2439aaf65860bcdae04f0', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 24 09:09:09.594203 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 24 09:09:09.594203 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Aug 24 09:09:09.594203 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 09:09:09.594203 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Aug 24 09:09:09.594203 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:09:09.594203 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:09:09.594203 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:09:09.594203 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:09:09.596148 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:09:09.596148 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:09:09.596148 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:09:09.596148 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:09:09.596148 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:09:09.596148 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:09:09.596148 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:09:09.596148 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Aug 24 09:09:09.596148 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Aug 24 09:09:09.596148 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:09:09.596148 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:09:09.596148 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 09:09:09.596148 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:09:09.596148 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 09:09:09.596148 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:09:09.596148 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Aug 24 09:09:09.596148 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Aug 24 09:09:09.599242 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Aug 24 09:09:09.599242 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Aug 24 09:09:09.599242 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 24 09:09:09.599242 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 24 09:09:09.599242 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 24 09:09:09.599242 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Aug 24 09:09:09.601239 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 483/965] 10.222.0.16 () {60 vars in 1131 bytes} [Wed Aug 24 09:09:09 2022] POST /identity/v3/users => generated 160 bytes in 242 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:09:09.607510 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138869e5-d9a0-4569-af52-41a5bbf42f0d None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:09.708879 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138869e5-d9a0-4569-af52-41a5bbf42f0d None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:09.744820 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2ff160-06dc-4259-aff5-ff58917d842d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:09.745779 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2ff160-06dc-4259-aff5-ff58917d842d None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:09.746337 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2ff160-06dc-4259-aff5-ff58917d842d None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:09.746825 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2ff160-06dc-4259-aff5-ff58917d842d None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:09.754325 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2ff160-06dc-4259-aff5-ff58917d842d None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:09.756016 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2ff160-06dc-4259-aff5-ff58917d842d None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:09.768343 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-8d2ff160-06dc-4259-aff5-ff58917d842d None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 09:09:09.768343 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:09:09.768343 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:09:09.768343 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:09:09.768343 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:09:09.768343 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:09:09.768343 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:09:09.768343 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:09:09.768343 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:09:09.768343 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:09:09.768343 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:09:09.768343 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:09:09.768343 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 09:09:09.768343 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 09:09:09.768343 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 09:09:09.768343 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 09:09:09.769391 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:09:09.769391 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:09:09.769391 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 09:09:09.769391 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:09:09.769391 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 09:09:09.769391 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:09:09.769391 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:09:09.769391 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:09:09.769391 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:09:09.769391 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:09:09.769391 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:09:09.769391 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 24 09:09:09.769391 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:09:09.769391 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:09:09.769391 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:09:09.769391 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:09:09.769391 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:09:09.769391 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:09:09.770400 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:09:09.770400 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:09:09.770400 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 09:09:09.770400 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 09:09:09.770400 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 09:09:09.770400 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 09:09:09.770400 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 09:09:09.770400 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 09:09:09.770400 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 09:09:09.770400 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 24 09:09:09.772955 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 483/966] 10.222.0.16 () {56 vars in 1099 bytes} [Wed Aug 24 09:09:09 2022] GET /identity/v3/users/cinder => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:09:09.779357 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a9f7cb-a467-4f0c-b32d-30609e2c719e None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:09.791853 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:09.839196 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2ff160-06dc-4259-aff5-ff58917d842d None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:09.851905 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:09.896573 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a9f7cb-a467-4f0c-b32d-30609e2c719e None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:09.942316 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bc865d-bc67-4180-924a-2a295c527acc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:09.943410 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bc865d-bc67-4180-924a-2a295c527acc None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:09.944085 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bc865d-bc67-4180-924a-2a295c527acc None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:09.944469 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bc865d-bc67-4180-924a-2a295c527acc None admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:09.945281 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bc865d-bc67-4180-924a-2a295c527acc None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:09:09.945863 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bc865d-bc67-4180-924a-2a295c527acc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:09.948273 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bc865d-bc67-4180-924a-2a295c527acc None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:09.952964 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2ff160-06dc-4259-aff5-ff58917d842d None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:09.989891 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e9f278-5061-4f76-9a07-e4e03035ba7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:09.991285 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e9f278-5061-4f76-9a07-e4e03035ba7a None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:09.991871 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e9f278-5061-4f76-9a07-e4e03035ba7a None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:09.992418 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e9f278-5061-4f76-9a07-e4e03035ba7a None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:09.997966 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 484/967] 10.222.0.16 () {56 vars in 1101 bytes} [Wed Aug 24 09:09:09 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:09:10.006973 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdcb6b7-9475-493c-a954-bf4656ed6003 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:10.007259 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdcb6b7-9475-493c-a954-bf4656ed6003 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:10.007777 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdcb6b7-9475-493c-a954-bf4656ed6003 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:10.009399 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e9f278-5061-4f76-9a07-e4e03035ba7a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:09:10.011799 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 485/968] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:09:10 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:09:10.015259 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102f877e-00ec-4b7e-889b-eadb967beabe None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:10.015544 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102f877e-00ec-4b7e-889b-eadb967beabe None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:10.015855 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102f877e-00ec-4b7e-889b-eadb967beabe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:10.016310 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-15e9f278-5061-4f76-9a07-e4e03035ba7a None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 09:09:10.016310 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:09:10.016310 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:09:10.016310 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:09:10.016310 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:09:10.016310 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:09:10.016310 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:09:10.016310 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:09:10.016310 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:09:10.016310 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:09:10.016310 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:09:10.016310 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:09:10.016310 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:09:10.016310 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:09:10.016310 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:09:10.016310 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:09:10.017250 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:09:10.017250 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:09:10.017250 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:09:10.017250 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:09:10.017250 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:09:10.017250 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:09:10.017250 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:09:10.017250 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:09:10.017250 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:09:10.017250 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:09:10.017250 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:09:10.017250 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 24 09:09:10.017250 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:09:10.017250 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:09:10.017250 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:09:10.017250 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:09:10.018127 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:09:10.018127 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:09:10.018127 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:09:10.018127 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:09:10.018127 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:09:10.018127 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:09:10.018127 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:09:10.018127 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:09:10.018127 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:09:10.018127 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:09:10.018127 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 09:09:10.018127 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 24 09:09:10.020457 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 484/969] 10.222.0.16 () {56 vars in 1120 bytes} [Wed Aug 24 09:09:09 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 09:09:10.026625 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e9f278-5061-4f76-9a07-e4e03035ba7a None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:10.036004 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-102f877e-00ec-4b7e-889b-eadb967beabe None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:09:10.037761 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-102f877e-00ec-4b7e-889b-eadb967beabe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:09:10.107932 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:10.129469 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:10.144667 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 486/970] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:09:10 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 09:09:10.155071 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e9f278-5061-4f76-9a07-e4e03035ba7a None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:10.191770 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605cfb47-b80c-48e3-b4df-ab7e16cfac5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:10.192765 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605cfb47-b80c-48e3-b4df-ab7e16cfac5f None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:10.193246 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605cfb47-b80c-48e3-b4df-ab7e16cfac5f None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:10.193720 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605cfb47-b80c-48e3-b4df-ab7e16cfac5f None admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:10.194664 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605cfb47-b80c-48e3-b4df-ab7e16cfac5f None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:09:10.195059 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605cfb47-b80c-48e3-b4df-ab7e16cfac5f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:10.196582 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605cfb47-b80c-48e3-b4df-ab7e16cfac5f None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:10.205835 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 485/971] 10.222.0.16 () {56 vars in 1115 bytes} [Wed Aug 24 09:09:10 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:09:10.212456 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102f877e-00ec-4b7e-889b-eadb967beabe None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:10.328231 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102f877e-00ec-4b7e-889b-eadb967beabe None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:10.363356 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657129ec-254f-4637-bf3f-4a7bc1b14142 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:10.364069 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657129ec-254f-4637-bf3f-4a7bc1b14142 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:10.364460 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657129ec-254f-4637-bf3f-4a7bc1b14142 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:10.364827 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657129ec-254f-4637-bf3f-4a7bc1b14142 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:10.375023 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657129ec-254f-4637-bf3f-4a7bc1b14142 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:10.376586 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657129ec-254f-4637-bf3f-4a7bc1b14142 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:10.392121 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-657129ec-254f-4637-bf3f-4a7bc1b14142 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:09:10.392121 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:09:10.392121 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:09:10.392121 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:09:10.392121 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:09:10.392121 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:09:10.392121 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:09:10.392121 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:09:10.392121 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:09:10.392121 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:09:10.392121 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:09:10.392121 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:09:10.392121 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:09:10.392121 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:09:10.392121 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:09:10.392121 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:09:10.393088 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:09:10.393088 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:09:10.393088 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:09:10.393088 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:09:10.393088 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:09:10.393088 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:09:10.393088 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:09:10.393088 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Aug 24 09:09:10.393088 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:09:10.393088 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:09:10.393088 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:09:10.393088 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:09:10.393088 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:09:10.393088 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:09:10.393088 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:09:10.393088 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:09:10.393088 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:09:10.394015 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:09:10.394015 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:09:10.394015 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Aug 24 09:09:10.394713 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 487/972] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 24 09:09:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:09:10.399846 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605cfb47-b80c-48e3-b4df-ab7e16cfac5f None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:10.409936 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657129ec-254f-4637-bf3f-4a7bc1b14142 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:10.464742 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:10.510633 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605cfb47-b80c-48e3-b4df-ab7e16cfac5f None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:10.516310 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657129ec-254f-4637-bf3f-4a7bc1b14142 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:10.538130 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:10.544265 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7171ded2-9b38-4739-b213-d78a1720d0d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:10.549371 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7171ded2-9b38-4739-b213-d78a1720d0d6 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:10.549371 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7171ded2-9b38-4739-b213-d78a1720d0d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:10.549371 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7171ded2-9b38-4739-b213-d78a1720d0d6 None admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:10.549371 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7171ded2-9b38-4739-b213-d78a1720d0d6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:09:10.549371 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7171ded2-9b38-4739-b213-d78a1720d0d6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:10.549371 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7171ded2-9b38-4739-b213-d78a1720d0d6 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:10.549861 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a07677-8791-42fd-8d70-75389f73682e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:10.550073 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a07677-8791-42fd-8d70-75389f73682e None admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:10.550354 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a07677-8791-42fd-8d70-75389f73682e None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:10.550629 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a07677-8791-42fd-8d70-75389f73682e None admin] PATH_INFO: `/v3/services` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:10.551265 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a07677-8791-42fd-8d70-75389f73682e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:10.552655 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a07677-8791-42fd-8d70-75389f73682e None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:10.559963 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 486/973] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Aug 24 09:09:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:09:10.561164 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a0a8d4-c12f-4300-b773-295e4a9a9092 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:10.561448 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a0a8d4-c12f-4300-b773-295e4a9a9092 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:10.561726 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a0a8d4-c12f-4300-b773-295e4a9a9092 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:10.562923 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 487/974] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:09:10 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:09:10.567679 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a0a8d4-c12f-4300-b773-295e4a9a9092 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:10.617929 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 488/975] 10.222.0.16 () {60 vars in 1141 bytes} [Wed Aug 24 09:09:10 2022] POST /identity/v3/services => generated 242 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 09:09:10.619836 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7807be-a23c-4d16-8a91-444a04f2ac9e None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:10.620149 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7807be-a23c-4d16-8a91-444a04f2ac9e None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:10.620421 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7807be-a23c-4d16-8a91-444a04f2ac9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:10.639289 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-dd7807be-a23c-4d16-8a91-444a04f2ac9e None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:09:10.644424 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-dd7807be-a23c-4d16-8a91-444a04f2ac9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:09:10.696598 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a0a8d4-c12f-4300-b773-295e4a9a9092 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:10.716672 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 489/976] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:09:10 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 09:09:10.735214 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa70f63-8b80-4de7-a9eb-2676d102c47f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:10.736146 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa70f63-8b80-4de7-a9eb-2676d102c47f None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:10.736595 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa70f63-8b80-4de7-a9eb-2676d102c47f None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:10.737014 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa70f63-8b80-4de7-a9eb-2676d102c47f None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:10.745796 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa70f63-8b80-4de7-a9eb-2676d102c47f None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:10.747521 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa70f63-8b80-4de7-a9eb-2676d102c47f None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:10.760675 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-2aa70f63-8b80-4de7-a9eb-2676d102c47f None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 09:09:10.760675 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:09:10.760675 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:09:10.760675 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:09:10.760675 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:09:10.760675 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:09:10.760675 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:09:10.760675 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:09:10.760675 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:09:10.760675 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:09:10.760675 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:09:10.760675 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:09:10.760675 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 09:09:10.760675 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 09:09:10.760675 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 09:09:10.760675 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 09:09:10.761804 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:09:10.761804 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:09:10.761804 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 09:09:10.761804 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:09:10.761804 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 09:09:10.761804 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:09:10.761804 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:09:10.761804 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:09:10.761804 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:09:10.761804 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:09:10.761804 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:09:10.761804 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 24 09:09:10.761804 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:09:10.761804 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:09:10.761804 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:09:10.761804 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:09:10.761804 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:09:10.761804 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:09:10.762753 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:09:10.762753 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:09:10.762753 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 09:09:10.762753 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 09:09:10.762753 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 09:09:10.762753 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 09:09:10.762753 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 09:09:10.762753 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 09:09:10.762753 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 09:09:10.762753 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 24 09:09:10.764759 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 488/977] 10.222.0.16 () {56 vars in 1117 bytes} [Wed Aug 24 09:09:10 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:09:10.770728 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7807be-a23c-4d16-8a91-444a04f2ac9e None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:10.815712 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:10.875440 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7807be-a23c-4d16-8a91-444a04f2ac9e None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:10.910222 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4420e89-a85a-4ca4-ae80-938c174fc72a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:10.910970 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4420e89-a85a-4ca4-ae80-938c174fc72a None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:10.911366 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4420e89-a85a-4ca4-ae80-938c174fc72a None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:10.911771 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4420e89-a85a-4ca4-ae80-938c174fc72a None admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:10.912594 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4420e89-a85a-4ca4-ae80-938c174fc72a None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:09:10.912997 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4420e89-a85a-4ca4-ae80-938c174fc72a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:10.914512 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4420e89-a85a-4ca4-ae80-938c174fc72a None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:10.936927 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 490/978] 10.222.0.16 () {56 vars in 1149 bytes} [Wed Aug 24 09:09:10 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:09:10.938492 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa70f63-8b80-4de7-a9eb-2676d102c47f None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:10.986773 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:11.004984 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4420e89-a85a-4ca4-ae80-938c174fc72a None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:11.045932 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa70f63-8b80-4de7-a9eb-2676d102c47f None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:11.081421 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5645d044-7128-459b-a9d3-c75f3748cc9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:11.082041 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5645d044-7128-459b-a9d3-c75f3748cc9a None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:11.082320 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5645d044-7128-459b-a9d3-c75f3748cc9a None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:11.083016 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5645d044-7128-459b-a9d3-c75f3748cc9a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:11.105955 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:11.112414 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4420e89-a85a-4ca4-ae80-938c174fc72a None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:11.159747 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0694b139-8334-4e84-8e3c-1104ec3681c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:11.159747 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0694b139-8334-4e84-8e3c-1104ec3681c6 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:11.159747 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0694b139-8334-4e84-8e3c-1104ec3681c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:11.159747 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0694b139-8334-4e84-8e3c-1104ec3681c6 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:11.160140 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0694b139-8334-4e84-8e3c-1104ec3681c6 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:11.160140 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0694b139-8334-4e84-8e3c-1104ec3681c6 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:11.160293 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5645d044-7128-459b-a9d3-c75f3748cc9a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:11.160293 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5645d044-7128-459b-a9d3-c75f3748cc9a None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:11.162276 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-0694b139-8334-4e84-8e3c-1104ec3681c6 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 09:09:11.162276 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:09:11.162276 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 09:09:11.162276 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 09:09:11.162276 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 09:09:11.162276 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 09:09:11.162276 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 09:09:11.162276 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:09:11.162276 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 09:09:11.162276 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Aug 24 09:09:11.162276 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 09:09:11.162276 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Aug 24 09:09:11.162276 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:09:11.162276 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:09:11.162276 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:09:11.162276 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:09:11.162276 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:09:11.163411 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:09:11.163411 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:09:11.163411 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:09:11.163411 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:09:11.163411 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:09:11.163411 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:09:11.163411 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 09:09:11.163411 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 09:09:11.163411 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 09:09:11.163411 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 09:09:11.163411 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:09:11.163411 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:09:11.163411 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:09:11.163411 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:09:11.163411 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:09:11.163411 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:09:11.164349 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:09:11.164349 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Aug 24 09:09:11.164349 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:09:11.164349 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:09:11.164349 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:09:11.164349 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:09:11.164349 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:09:11.164349 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:09:11.164349 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:09:11.164349 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:09:11.164349 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 09:09:11.164349 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:09:11.164349 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 09:09:11.164349 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Aug 24 09:09:11.164349 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 491/979] 10.222.0.16 () {56 vars in 1126 bytes} [Wed Aug 24 09:09:11 2022] GET /identity/v3/services/object-store => generated 93 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 09:09:11.168857 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0694b139-8334-4e84-8e3c-1104ec3681c6 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:11.211478 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:11.274003 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 489/980] 10.222.0.16 () {58 vars in 1282 bytes} [Wed Aug 24 09:09:10 2022] GET /identity/v3/auth/tokens => generated 1570 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 24 09:09:11.280254 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5645d044-7128-459b-a9d3-c75f3748cc9a None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:11.280835 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0694b139-8334-4e84-8e3c-1104ec3681c6 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:11.320114 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7de5f3-c7ec-40ca-8176-4c45dd5f4628 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:11.320727 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7de5f3-c7ec-40ca-8176-4c45dd5f4628 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:11.321527 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7de5f3-c7ec-40ca-8176-4c45dd5f4628 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:11.321818 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7de5f3-c7ec-40ca-8176-4c45dd5f4628 None admin] PATH_INFO: `/v3/services` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:11.322452 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7de5f3-c7ec-40ca-8176-4c45dd5f4628 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:09:11.322758 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7de5f3-c7ec-40ca-8176-4c45dd5f4628 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:11.324155 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7de5f3-c7ec-40ca-8176-4c45dd5f4628 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:11.331635 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 492/981] 10.222.0.16 () {56 vars in 1122 bytes} [Wed Aug 24 09:09:11 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:09:11.334959 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7de5f3-c7ec-40ca-8176-4c45dd5f4628 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:11.361019 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:11.392306 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5645d044-7128-459b-a9d3-c75f3748cc9a None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:11.428408 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c1d135-fe8d-4d87-8078-727fcbf70ec8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:11.428961 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c1d135-fe8d-4d87-8078-727fcbf70ec8 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:11.429254 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c1d135-fe8d-4d87-8078-727fcbf70ec8 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:11.429567 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c1d135-fe8d-4d87-8078-727fcbf70ec8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:11.440409 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c1d135-fe8d-4d87-8078-727fcbf70ec8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:11.441841 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c1d135-fe8d-4d87-8078-727fcbf70ec8 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:11.444300 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7de5f3-c7ec-40ca-8176-4c45dd5f4628 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:11.449306 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:11.457371 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-a9c1d135-fe8d-4d87-8078-727fcbf70ec8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:09:11.457371 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:09:11.457371 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:09:11.457371 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:09:11.457371 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:09:11.457371 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:09:11.457371 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:09:11.457371 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:09:11.457371 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:09:11.457371 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:09:11.457371 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:09:11.457371 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:09:11.457371 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:09:11.457371 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:09:11.457371 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:09:11.457371 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:09:11.458803 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:09:11.458803 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:09:11.458803 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:09:11.458803 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:09:11.458803 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:09:11.458803 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:09:11.458803 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:09:11.458803 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 24 09:09:11.458803 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:09:11.458803 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:09:11.458803 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:09:11.458803 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:09:11.458803 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:09:11.458803 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:09:11.458803 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:09:11.458803 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:09:11.458803 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:09:11.460024 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:09:11.460024 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:09:11.460024 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 24 09:09:11.460024 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 490/982] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 24 09:09:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:09:11.464535 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c1d135-fe8d-4d87-8078-727fcbf70ec8 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:11.484495 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e64a4a-0838-43f7-b317-c45f1a4f9bc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:11.485369 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e64a4a-0838-43f7-b317-c45f1a4f9bc7 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:11.485651 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e64a4a-0838-43f7-b317-c45f1a4f9bc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:11.485927 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e64a4a-0838-43f7-b317-c45f1a4f9bc7 None admin] PATH_INFO: `/v3/services` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:11.486560 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e64a4a-0838-43f7-b317-c45f1a4f9bc7 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:09:11.486868 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e64a4a-0838-43f7-b317-c45f1a4f9bc7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:11.488293 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e64a4a-0838-43f7-b317-c45f1a4f9bc7 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:11.494476 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 493/983] 10.222.0.16 () {56 vars in 1122 bytes} [Wed Aug 24 09:09:11 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:09:11.500073 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e64a4a-0838-43f7-b317-c45f1a4f9bc7 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:11.571315 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c1d135-fe8d-4d87-8078-727fcbf70ec8 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:11.596435 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:11.608697 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b15eae-12c9-4b3c-ae18-4e4f1fc0af7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:11.609314 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b15eae-12c9-4b3c-ae18-4e4f1fc0af7e None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:11.609591 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b15eae-12c9-4b3c-ae18-4e4f1fc0af7e None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:11.609732 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e64a4a-0838-43f7-b317-c45f1a4f9bc7 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:11.609918 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b15eae-12c9-4b3c-ae18-4e4f1fc0af7e None admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:11.610597 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b15eae-12c9-4b3c-ae18-4e4f1fc0af7e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:09:11.610899 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b15eae-12c9-4b3c-ae18-4e4f1fc0af7e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:11.612634 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b15eae-12c9-4b3c-ae18-4e4f1fc0af7e None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:11.624970 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 491/984] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Aug 24 09:09:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:09:11.630399 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b15eae-12c9-4b3c-ae18-4e4f1fc0af7e None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:11.644992 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d67075b-b2ec-4edd-b7ac-9183ef5e581a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:11.645604 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d67075b-b2ec-4edd-b7ac-9183ef5e581a None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:11.645871 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d67075b-b2ec-4edd-b7ac-9183ef5e581a None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:11.646143 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d67075b-b2ec-4edd-b7ac-9183ef5e581a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:11.646853 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d67075b-b2ec-4edd-b7ac-9183ef5e581a None admin] RBAC: Adding query filter params (interface=admin, service_id=f0445fbd21444c45a7931845d16b4809, region_id=RegionOne) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:09:11.647158 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d67075b-b2ec-4edd-b7ac-9183ef5e581a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:11.648538 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d67075b-b2ec-4edd-b7ac-9183ef5e581a None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:11.654682 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 494/985] 10.222.0.16 () {56 vars in 1249 bytes} [Wed Aug 24 09:09:11 2022] GET /identity/v3/endpoints?service_id=f0445fbd21444c45a7931845d16b4809&interface=admin®ion_id=RegionOne => generated 193 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:09:11.659962 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d67075b-b2ec-4edd-b7ac-9183ef5e581a None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:11.696872 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:11.743047 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b15eae-12c9-4b3c-ae18-4e4f1fc0af7e None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:11.770297 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d67075b-b2ec-4edd-b7ac-9183ef5e581a None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:11.779288 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3ce550-b7d3-4c11-9f3d-01deb720b803 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:11.779948 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3ce550-b7d3-4c11-9f3d-01deb720b803 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:11.780224 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3ce550-b7d3-4c11-9f3d-01deb720b803 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:11.780492 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3ce550-b7d3-4c11-9f3d-01deb720b803 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:11.787354 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3ce550-b7d3-4c11-9f3d-01deb720b803 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:11.788827 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3ce550-b7d3-4c11-9f3d-01deb720b803 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:11.800314 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-bb3ce550-b7d3-4c11-9f3d-01deb720b803 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:09:11.800314 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:09:11.800314 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:09:11.800314 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:09:11.800314 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:09:11.800314 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:09:11.800314 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:09:11.800314 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:09:11.800314 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:09:11.800314 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:09:11.800314 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:09:11.800314 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:09:11.800314 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 09:09:11.800314 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 09:09:11.800314 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 09:09:11.801413 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 09:09:11.801413 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:09:11.801413 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:09:11.801413 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:09:11.801413 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:09:11.801413 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:09:11.801413 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:09:11.801413 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:09:11.801413 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 24 09:09:11.801413 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:09:11.801413 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:09:11.801413 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:09:11.801413 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:09:11.801413 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:09:11.801413 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:09:11.801413 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:09:11.801413 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:09:11.802372 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 09:09:11.802372 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 09:09:11.802372 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 09:09:11.802372 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 09:09:11.802372 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 09:09:11.802372 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 09:09:11.802372 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:09:11.802372 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 24 09:09:11.802372 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 492/986] 10.222.0.16 () {56 vars in 1111 bytes} [Wed Aug 24 09:09:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 09:09:11.807129 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3ce550-b7d3-4c11-9f3d-01deb720b803 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:11.809417 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84dcf91-4455-4280-85eb-0d7e0b87805b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:11.810202 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84dcf91-4455-4280-85eb-0d7e0b87805b None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:11.811317 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84dcf91-4455-4280-85eb-0d7e0b87805b None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:11.811627 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84dcf91-4455-4280-85eb-0d7e0b87805b None admin] PATH_INFO: `/v3/services` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:11.812248 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84dcf91-4455-4280-85eb-0d7e0b87805b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:11.813617 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84dcf91-4455-4280-85eb-0d7e0b87805b None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:11.831613 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 495/987] 10.222.0.16 () {56 vars in 1087 bytes} [Wed Aug 24 09:09:11 2022] GET /identity/v3/services => generated 1727 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 09:09:11.915152 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3ce550-b7d3-4c11-9f3d-01deb720b803 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:11.933954 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:11.955258 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23affc8a-7fcb-456d-929a-cc8b48ccd6be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:11.955258 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23affc8a-7fcb-456d-929a-cc8b48ccd6be None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:11.955258 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23affc8a-7fcb-456d-929a-cc8b48ccd6be None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:11.955258 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23affc8a-7fcb-456d-929a-cc8b48ccd6be None admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:11.955875 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23affc8a-7fcb-456d-929a-cc8b48ccd6be None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:09:11.955968 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23affc8a-7fcb-456d-929a-cc8b48ccd6be None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:11.957371 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23affc8a-7fcb-456d-929a-cc8b48ccd6be None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:11.970590 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 493/988] 10.222.0.16 () {56 vars in 1148 bytes} [Wed Aug 24 09:09:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:09:11.976696 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84dcf91-4455-4280-85eb-0d7e0b87805b None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:12.003653 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:12.080820 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84dcf91-4455-4280-85eb-0d7e0b87805b None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:12.114926 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07ab325-de3a-4948-8416-a30826ead772 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:12.115542 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07ab325-de3a-4948-8416-a30826ead772 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:12.115854 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07ab325-de3a-4948-8416-a30826ead772 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:12.116132 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07ab325-de3a-4948-8416-a30826ead772 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:12.116865 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07ab325-de3a-4948-8416-a30826ead772 None admin] RBAC: Adding query filter params (role.id=716e298774d64bc89a265236b7b9db7f, scope.project.id=c467e88e2fb0491bacefcf6d2ce326d9, user.id=0489462287fc4aea8e8e720af24e1ef8) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:09:12.117172 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07ab325-de3a-4948-8416-a30826ead772 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:12.118636 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07ab325-de3a-4948-8416-a30826ead772 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:12.124583 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 496/989] 10.222.0.16 () {56 vars in 1374 bytes} [Wed Aug 24 09:09:11 2022] GET /identity/v3/role_assignments?user.id=0489462287fc4aea8e8e720af24e1ef8&scope.project.id=c467e88e2fb0491bacefcf6d2ce326d9&role.id=716e298774d64bc89a265236b7b9db7f => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:09:12.553258 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cfab13-fc6f-4a23-ace7-f1cdca0f1e64 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:12.553761 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cfab13-fc6f-4a23-ace7-f1cdca0f1e64 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:12.553761 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cfab13-fc6f-4a23-ace7-f1cdca0f1e64 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:12.559301 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 494/990] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:09:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:09:12.560900 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eea69bc-40b4-4cf9-9dc2-779ed6d7ec6e None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:12.561191 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eea69bc-40b4-4cf9-9dc2-779ed6d7ec6e None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:12.561465 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eea69bc-40b4-4cf9-9dc2-779ed6d7ec6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:12.580203 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-8eea69bc-40b4-4cf9-9dc2-779ed6d7ec6e None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:09:12.581385 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-8eea69bc-40b4-4cf9-9dc2-779ed6d7ec6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:09:12.649149 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 497/991] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:09:12 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 09:09:12.914512 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cfab13-fc6f-4a23-ace7-f1cdca0f1e64 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:13.011893 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cfab13-fc6f-4a23-ace7-f1cdca0f1e64 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:13.045331 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b2a776-abd7-4462-86e6-209ccb8379a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:13.045949 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b2a776-abd7-4462-86e6-209ccb8379a5 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:13.046225 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b2a776-abd7-4462-86e6-209ccb8379a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:13.046494 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b2a776-abd7-4462-86e6-209ccb8379a5 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:13.047067 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b2a776-abd7-4462-86e6-209ccb8379a5 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:13.048445 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b2a776-abd7-4462-86e6-209ccb8379a5 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:13.056916 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-a1b2a776-abd7-4462-86e6-209ccb8379a5 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 09:09:13.056916 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:09:13.056916 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 09:09:13.056916 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 09:09:13.056916 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 09:09:13.056916 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 09:09:13.056916 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 09:09:13.056916 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:09:13.056916 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 09:09:13.056916 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 24 09:09:13.056916 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 09:09:13.056916 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 24 09:09:13.056916 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:09:13.056916 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:09:13.056916 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:09:13.056916 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:09:13.056916 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:09:13.058128 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:09:13.058128 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:09:13.058128 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:09:13.058128 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:09:13.058128 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:09:13.058128 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:09:13.058128 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 09:09:13.058128 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 09:09:13.058128 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 09:09:13.058128 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 09:09:13.058128 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:09:13.058128 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:09:13.058128 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:09:13.058128 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:09:13.058128 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:09:13.058128 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:09:13.059009 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:09:13.059009 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 24 09:09:13.059009 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:09:13.059009 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:09:13.059009 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:09:13.059009 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:09:13.059009 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:09:13.059009 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:09:13.059009 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:09:13.059009 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:09:13.059009 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 09:09:13.059009 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:09:13.059009 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 09:09:13.059009 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 24 09:09:13.059009 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 495/992] 10.222.0.16 () {56 vars in 1111 bytes} [Wed Aug 24 09:09:12 2022] GET /identity/v3/services/compute => generated 88 bytes in 144 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:09:13.063920 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eea69bc-40b4-4cf9-9dc2-779ed6d7ec6e None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:13.075776 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:13.164600 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eea69bc-40b4-4cf9-9dc2-779ed6d7ec6e None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:13.200914 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc67cdb-a934-453b-b785-5615e51096f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:13.201888 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc67cdb-a934-453b-b785-5615e51096f9 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:13.202360 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc67cdb-a934-453b-b785-5615e51096f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:13.202817 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc67cdb-a934-453b-b785-5615e51096f9 None admin] PATH_INFO: `/v3/services` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:13.203926 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc67cdb-a934-453b-b785-5615e51096f9 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:09:13.204443 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc67cdb-a934-453b-b785-5615e51096f9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:13.205991 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc67cdb-a934-453b-b785-5615e51096f9 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:13.211379 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 498/993] 10.222.0.16 () {56 vars in 1112 bytes} [Wed Aug 24 09:09:13 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:09:13.217353 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b2a776-abd7-4462-86e6-209ccb8379a5 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:13.263847 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:13.329083 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b2a776-abd7-4462-86e6-209ccb8379a5 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:13.364315 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e24c19-cb62-4c8d-b2ef-810fda99224c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:13.364924 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e24c19-cb62-4c8d-b2ef-810fda99224c None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:13.365204 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e24c19-cb62-4c8d-b2ef-810fda99224c None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:13.365476 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e24c19-cb62-4c8d-b2ef-810fda99224c None admin] PATH_INFO: `/v3/services` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:13.366103 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e24c19-cb62-4c8d-b2ef-810fda99224c None admin] RBAC: Adding query filter params (type=compute) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:09:13.366405 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e24c19-cb62-4c8d-b2ef-810fda99224c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:13.367797 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e24c19-cb62-4c8d-b2ef-810fda99224c None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:13.372866 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 496/994] 10.222.0.16 () {56 vars in 1112 bytes} [Wed Aug 24 09:09:13 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:09:13.378557 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc67cdb-a934-453b-b785-5615e51096f9 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:13.458671 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:13.484591 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc67cdb-a934-453b-b785-5615e51096f9 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:13.519721 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d55852f-870a-4f2a-9d9c-bb6ed11e2bbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:13.520340 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d55852f-870a-4f2a-9d9c-bb6ed11e2bbf None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:13.520705 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d55852f-870a-4f2a-9d9c-bb6ed11e2bbf None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:13.520984 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d55852f-870a-4f2a-9d9c-bb6ed11e2bbf None admin] PATH_INFO: `/v3/endpoints` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:13.521689 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d55852f-870a-4f2a-9d9c-bb6ed11e2bbf None admin] RBAC: Adding query filter params (interface=public, service_id=600173edd4f14eb29454e7df59f2ea57, region_id=RegionOne) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:09:13.522000 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d55852f-870a-4f2a-9d9c-bb6ed11e2bbf None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:13.523371 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d55852f-870a-4f2a-9d9c-bb6ed11e2bbf None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:13.528622 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 499/995] 10.222.0.16 () {56 vars in 1251 bytes} [Wed Aug 24 09:09:13 2022] GET /identity/v3/endpoints?service_id=600173edd4f14eb29454e7df59f2ea57&interface=public®ion_id=RegionOne => generated 194 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:09:13.533976 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e24c19-cb62-4c8d-b2ef-810fda99224c None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:13.636309 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e24c19-cb62-4c8d-b2ef-810fda99224c None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:13.643361 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:13.670838 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c5fc73-b4e5-4bef-9290-9cf7240ce81f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:13.671446 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c5fc73-b4e5-4bef-9290-9cf7240ce81f None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:13.671756 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c5fc73-b4e5-4bef-9290-9cf7240ce81f None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:13.672031 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c5fc73-b4e5-4bef-9290-9cf7240ce81f None admin] PATH_INFO: `/v3/services` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:13.672668 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c5fc73-b4e5-4bef-9290-9cf7240ce81f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:13.674297 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c5fc73-b4e5-4bef-9290-9cf7240ce81f None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:13.681739 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 497/996] 10.222.0.16 () {56 vars in 1087 bytes} [Wed Aug 24 09:09:13 2022] GET /identity/v3/services => generated 1727 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:09:13.750363 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d56ed1-d254-4d32-94ed-2eb7821a3c3b None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:13.750557 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d56ed1-d254-4d32-94ed-2eb7821a3c3b None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:13.750840 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d56ed1-d254-4d32-94ed-2eb7821a3c3b None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:13.752403 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 500/997] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:09:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:09:13.758155 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd05e0e4-1e2f-4c59-a0dd-dbedfd299942 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:13.758821 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd05e0e4-1e2f-4c59-a0dd-dbedfd299942 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:13.759110 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd05e0e4-1e2f-4c59-a0dd-dbedfd299942 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:13.778845 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-dd05e0e4-1e2f-4c59-a0dd-dbedfd299942 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:09:13.780108 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-dd05e0e4-1e2f-4c59-a0dd-dbedfd299942 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:09:13.856945 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 498/998] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:09:13 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 09:09:14.100871 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e39944-e619-4279-90ce-08c7f6ac6e66 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:14.101279 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e39944-e619-4279-90ce-08c7f6ac6e66 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:14.101459 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e39944-e619-4279-90ce-08c7f6ac6e66 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:14.103465 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 501/999] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:09:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:09:14.108482 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0489efd-c905-4222-938a-65c06491df8f None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:14.108767 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0489efd-c905-4222-938a-65c06491df8f None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:14.109035 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0489efd-c905-4222-938a-65c06491df8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:14.124207 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e39944-e619-4279-90ce-08c7f6ac6e66 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:14.126985 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-a0489efd-c905-4222-938a-65c06491df8f None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:09:14.128201 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-a0489efd-c905-4222-938a-65c06491df8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:09:14.202560 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 499/1000] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:09:14 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 09:09:14.233706 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e39944-e619-4279-90ce-08c7f6ac6e66 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:14.268123 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca59e027-81d8-4a41-9078-dd594cfda999 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:14.268747 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca59e027-81d8-4a41-9078-dd594cfda999 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:14.269025 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca59e027-81d8-4a41-9078-dd594cfda999 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:14.269300 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca59e027-81d8-4a41-9078-dd594cfda999 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:14.269873 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca59e027-81d8-4a41-9078-dd594cfda999 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:14.271232 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca59e027-81d8-4a41-9078-dd594cfda999 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:14.280492 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-ca59e027-81d8-4a41-9078-dd594cfda999 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 09:09:14.280492 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:09:14.280492 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 09:09:14.280492 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 09:09:14.280492 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 09:09:14.280492 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 09:09:14.280492 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 09:09:14.280492 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:09:14.280492 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 09:09:14.280492 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Aug 24 09:09:14.280492 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 09:09:14.280492 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Aug 24 09:09:14.280492 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:09:14.280492 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:09:14.280492 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:09:14.280492 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:09:14.280492 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:09:14.281477 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:09:14.281477 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:09:14.281477 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:09:14.281477 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:09:14.281477 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:09:14.281477 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:09:14.281477 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 09:09:14.281477 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 09:09:14.281477 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 09:09:14.281477 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 09:09:14.281477 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:09:14.281477 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:09:14.281477 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:09:14.281477 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:09:14.281477 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:09:14.281477 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:09:14.282314 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:09:14.282314 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Aug 24 09:09:14.282314 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:09:14.282314 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:09:14.282314 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:09:14.282314 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:09:14.282314 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:09:14.282314 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:09:14.282314 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:09:14.282314 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:09:14.282314 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 09:09:14.282314 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:09:14.282314 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 09:09:14.282314 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Aug 24 09:09:14.282314 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 502/1001] 10.222.0.16 () {56 vars in 1126 bytes} [Wed Aug 24 09:09:14 2022] GET /identity/v3/services/object-store => generated 93 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 09:09:14.287338 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0489efd-c905-4222-938a-65c06491df8f None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:14.318069 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:14.388156 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0489efd-c905-4222-938a-65c06491df8f None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:14.422503 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171a2ae3-61a3-4e4b-990b-338bd596f9d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:14.423100 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171a2ae3-61a3-4e4b-990b-338bd596f9d7 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:14.423375 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171a2ae3-61a3-4e4b-990b-338bd596f9d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:14.423667 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171a2ae3-61a3-4e4b-990b-338bd596f9d7 None admin] PATH_INFO: `/v3/services` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:14.424288 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171a2ae3-61a3-4e4b-990b-338bd596f9d7 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:09:14.424586 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171a2ae3-61a3-4e4b-990b-338bd596f9d7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:14.425935 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171a2ae3-61a3-4e4b-990b-338bd596f9d7 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:14.431357 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 500/1002] 10.222.0.16 () {56 vars in 1122 bytes} [Wed Aug 24 09:09:14 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:09:14.436853 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca59e027-81d8-4a41-9078-dd594cfda999 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:14.439777 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:14.480032 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171a2ae3-61a3-4e4b-990b-338bd596f9d7 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:14.539603 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca59e027-81d8-4a41-9078-dd594cfda999 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:14.574015 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:14.579444 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51f8512-0370-4ad8-80a1-9f2692e99bd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:14.580097 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51f8512-0370-4ad8-80a1-9f2692e99bd2 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:14.580568 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51f8512-0370-4ad8-80a1-9f2692e99bd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:14.580848 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51f8512-0370-4ad8-80a1-9f2692e99bd2 None admin] PATH_INFO: `/v3/services` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:14.581251 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171a2ae3-61a3-4e4b-990b-338bd596f9d7 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:14.581482 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51f8512-0370-4ad8-80a1-9f2692e99bd2 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:09:14.581783 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51f8512-0370-4ad8-80a1-9f2692e99bd2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:14.584167 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51f8512-0370-4ad8-80a1-9f2692e99bd2 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:14.590384 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 503/1003] 10.222.0.16 () {56 vars in 1122 bytes} [Wed Aug 24 09:09:14 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:09:14.595993 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51f8512-0370-4ad8-80a1-9f2692e99bd2 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:14.618732 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20d72ae-1797-4548-8cc1-5d964af077fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:14.619368 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20d72ae-1797-4548-8cc1-5d964af077fb None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:14.619682 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20d72ae-1797-4548-8cc1-5d964af077fb None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:14.619957 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20d72ae-1797-4548-8cc1-5d964af077fb None admin] PATH_INFO: `/v3/services/image` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:14.620540 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20d72ae-1797-4548-8cc1-5d964af077fb None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:14.621898 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20d72ae-1797-4548-8cc1-5d964af077fb None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:14.630698 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-d20d72ae-1797-4548-8cc1-5d964af077fb None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 09:09:14.630698 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:09:14.630698 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 09:09:14.630698 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 09:09:14.630698 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 09:09:14.630698 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 09:09:14.630698 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 09:09:14.630698 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:09:14.630698 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 09:09:14.630698 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 24 09:09:14.630698 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 09:09:14.630698 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 24 09:09:14.630698 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:09:14.630698 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:09:14.630698 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:09:14.630698 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:09:14.630698 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:09:14.631945 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:09:14.631945 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:09:14.631945 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:09:14.631945 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:09:14.631945 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:09:14.631945 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:09:14.631945 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 09:09:14.631945 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 09:09:14.631945 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 09:09:14.631945 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 09:09:14.631945 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:09:14.631945 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:09:14.631945 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:09:14.631945 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:09:14.631945 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:09:14.631945 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:09:14.632838 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:09:14.632838 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 24 09:09:14.632838 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:09:14.632838 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:09:14.632838 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:09:14.632838 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:09:14.632838 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:09:14.632838 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:09:14.632838 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:09:14.632838 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:09:14.632838 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 09:09:14.632838 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:09:14.632838 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 09:09:14.632838 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 24 09:09:14.632838 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 501/1004] 10.222.0.16 () {56 vars in 1105 bytes} [Wed Aug 24 09:09:14 2022] GET /identity/v3/services/image => generated 86 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 09:09:14.637974 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20d72ae-1797-4548-8cc1-5d964af077fb None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:14.671614 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:14.706749 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51f8512-0370-4ad8-80a1-9f2692e99bd2 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:14.742137 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b0eb66-45ef-4bd3-80c9-01f1a6b6889e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:14.743011 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b0eb66-45ef-4bd3-80c9-01f1a6b6889e None admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:14.743465 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b0eb66-45ef-4bd3-80c9-01f1a6b6889e None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:14.743909 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b0eb66-45ef-4bd3-80c9-01f1a6b6889e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:14.744799 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b0eb66-45ef-4bd3-80c9-01f1a6b6889e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:14.746193 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20d72ae-1797-4548-8cc1-5d964af077fb None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:14.746500 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b0eb66-45ef-4bd3-80c9-01f1a6b6889e None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:14.781424 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffede001-c0c0-464b-a301-326e972f0668 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:14.782025 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffede001-c0c0-464b-a301-326e972f0668 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:14.782303 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffede001-c0c0-464b-a301-326e972f0668 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:14.782572 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffede001-c0c0-464b-a301-326e972f0668 None admin] PATH_INFO: `/v3/services` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:14.783194 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffede001-c0c0-464b-a301-326e972f0668 None admin] RBAC: Adding query filter params (name=image) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:09:14.783494 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffede001-c0c0-464b-a301-326e972f0668 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:14.784878 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffede001-c0c0-464b-a301-326e972f0668 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:14.789468 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 502/1005] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 24 09:09:14 2022] GET /identity/v3/services?name=image => generated 122 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:09:14.796183 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffede001-c0c0-464b-a301-326e972f0668 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:14.820511 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:14.824589 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 504/1006] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 24 09:09:14 2022] POST /identity/v3/endpoints => generated 325 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 09:09:14.905006 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffede001-c0c0-464b-a301-326e972f0668 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:14.947279 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4238dc2b-8fff-42f0-b1ce-c986647fa871 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:14.947932 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4238dc2b-8fff-42f0-b1ce-c986647fa871 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:14.948280 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4238dc2b-8fff-42f0-b1ce-c986647fa871 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:14.948616 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4238dc2b-8fff-42f0-b1ce-c986647fa871 None admin] PATH_INFO: `/v3/services` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:14.949314 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4238dc2b-8fff-42f0-b1ce-c986647fa871 None admin] RBAC: Adding query filter params (type=image) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:09:14.949682 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4238dc2b-8fff-42f0-b1ce-c986647fa871 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:14.951112 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4238dc2b-8fff-42f0-b1ce-c986647fa871 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:14.963780 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 503/1007] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 24 09:09:14 2022] GET /identity/v3/services?type=image => generated 122 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:09:15.599311 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3015e92b-1a72-455e-ab08-d39ca1552dc1 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:15.599841 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3015e92b-1a72-455e-ab08-d39ca1552dc1 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:15.599841 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3015e92b-1a72-455e-ab08-d39ca1552dc1 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:15.601398 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 505/1008] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:09:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:09:15.607108 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a73da0-7292-4aff-a3f3-c0b04ab8adc4 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:15.607397 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a73da0-7292-4aff-a3f3-c0b04ab8adc4 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:15.607710 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a73da0-7292-4aff-a3f3-c0b04ab8adc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:15.625589 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-b5a73da0-7292-4aff-a3f3-c0b04ab8adc4 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:09:15.626777 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-b5a73da0-7292-4aff-a3f3-c0b04ab8adc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:09:15.692772 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 504/1009] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:09:15 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 09:09:15.970268 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3015e92b-1a72-455e-ab08-d39ca1552dc1 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:16.074245 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3015e92b-1a72-455e-ab08-d39ca1552dc1 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:16.099493 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:16.108866 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a666888-8b7f-451a-ae30-0bea99935e2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:16.109492 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a666888-8b7f-451a-ae30-0bea99935e2c None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:16.109769 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a666888-8b7f-451a-ae30-0bea99935e2c None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:16.110040 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a666888-8b7f-451a-ae30-0bea99935e2c None admin] PATH_INFO: `/v3/services/compute` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:16.115238 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a666888-8b7f-451a-ae30-0bea99935e2c None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:16.115238 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a666888-8b7f-451a-ae30-0bea99935e2c None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:16.120979 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-6a666888-8b7f-451a-ae30-0bea99935e2c None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 09:09:16.120979 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:09:16.120979 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 09:09:16.120979 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 09:09:16.120979 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 09:09:16.120979 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 09:09:16.120979 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 09:09:16.120979 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:09:16.120979 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 09:09:16.120979 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Aug 24 09:09:16.120979 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 09:09:16.120979 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Aug 24 09:09:16.120979 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:09:16.120979 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:09:16.120979 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:09:16.120979 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:09:16.120979 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:09:16.121958 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:09:16.121958 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:09:16.121958 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:09:16.121958 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:09:16.121958 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:09:16.121958 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:09:16.121958 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 09:09:16.121958 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 09:09:16.121958 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 09:09:16.121958 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 09:09:16.121958 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:09:16.121958 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:09:16.121958 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:09:16.121958 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:09:16.121958 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:09:16.121958 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:09:16.122847 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:09:16.122847 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Aug 24 09:09:16.122847 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:09:16.122847 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:09:16.122847 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:09:16.122847 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:09:16.122847 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:09:16.122847 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:09:16.122847 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:09:16.122847 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:09:16.122847 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 09:09:16.122847 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:09:16.122847 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 09:09:16.122847 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Aug 24 09:09:16.130248 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 506/1010] 10.222.0.16 () {56 vars in 1111 bytes} [Wed Aug 24 09:09:15 2022] GET /identity/v3/services/compute => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:09:16.130398 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a73da0-7292-4aff-a3f3-c0b04ab8adc4 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:16.144213 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:16.232393 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a73da0-7292-4aff-a3f3-c0b04ab8adc4 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:16.266971 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04228cea-520e-4787-9bf6-9db6e5574fe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:16.267606 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04228cea-520e-4787-9bf6-9db6e5574fe0 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:16.267886 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04228cea-520e-4787-9bf6-9db6e5574fe0 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:16.268160 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04228cea-520e-4787-9bf6-9db6e5574fe0 None admin] PATH_INFO: `/v3/services` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:16.268792 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04228cea-520e-4787-9bf6-9db6e5574fe0 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:09:16.269095 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04228cea-520e-4787-9bf6-9db6e5574fe0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:16.270455 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04228cea-520e-4787-9bf6-9db6e5574fe0 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:16.279794 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 505/1011] 10.222.0.16 () {56 vars in 1112 bytes} [Wed Aug 24 09:09:16 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:09:16.281976 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a666888-8b7f-451a-ae30-0bea99935e2c None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:16.390379 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a666888-8b7f-451a-ae30-0bea99935e2c None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:16.425602 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70902a8f-ec85-4691-bafe-87f7cf127f09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:16.426327 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70902a8f-ec85-4691-bafe-87f7cf127f09 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:16.426560 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70902a8f-ec85-4691-bafe-87f7cf127f09 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:16.426830 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70902a8f-ec85-4691-bafe-87f7cf127f09 None admin] PATH_INFO: `/v3/services` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:16.427514 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70902a8f-ec85-4691-bafe-87f7cf127f09 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:09:16.427853 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70902a8f-ec85-4691-bafe-87f7cf127f09 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:16.429275 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70902a8f-ec85-4691-bafe-87f7cf127f09 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:16.439627 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 507/1012] 10.222.0.16 () {56 vars in 1112 bytes} [Wed Aug 24 09:09:16 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:09:16.440894 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04228cea-520e-4787-9bf6-9db6e5574fe0 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:16.521443 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:16.545169 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04228cea-520e-4787-9bf6-9db6e5574fe0 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:16.580326 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ccdf3e-1b7e-4bc9-b6e3-5f81669b301b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:16.580936 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ccdf3e-1b7e-4bc9-b6e3-5f81669b301b None admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:16.581280 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ccdf3e-1b7e-4bc9-b6e3-5f81669b301b None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:16.581562 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ccdf3e-1b7e-4bc9-b6e3-5f81669b301b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:16.582224 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ccdf3e-1b7e-4bc9-b6e3-5f81669b301b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:16.583620 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ccdf3e-1b7e-4bc9-b6e3-5f81669b301b None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:16.691604 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 506/1013] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 24 09:09:16 2022] POST /identity/v3/endpoints => generated 334 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:09:16.804890 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c1a6fc-84d9-4e4c-9de9-1c1b633bfab2 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:16.805468 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c1a6fc-84d9-4e4c-9de9-1c1b633bfab2 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:16.805468 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c1a6fc-84d9-4e4c-9de9-1c1b633bfab2 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:16.807007 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 508/1014] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:09:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:09:16.812678 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d1152e-c9de-49d1-ae6d-8e168162dd88 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:16.813140 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d1152e-c9de-49d1-ae6d-8e168162dd88 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:16.813216 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d1152e-c9de-49d1-ae6d-8e168162dd88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:16.831559 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-17d1152e-c9de-49d1-ae6d-8e168162dd88 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:09:16.832757 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-17d1152e-c9de-49d1-ae6d-8e168162dd88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:09:16.844323 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856bac3d-2acc-4413-a4aa-e4f5923cb2bf None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:16.844602 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856bac3d-2acc-4413-a4aa-e4f5923cb2bf None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:16.844876 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856bac3d-2acc-4413-a4aa-e4f5923cb2bf None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:16.846191 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 509/1015] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:09:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:09:16.851519 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281db89e-3977-4cb6-928b-5ccecea8974b None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:16.851857 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281db89e-3977-4cb6-928b-5ccecea8974b None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:16.852730 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281db89e-3977-4cb6-928b-5ccecea8974b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:16.870662 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-281db89e-3977-4cb6-928b-5ccecea8974b None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:09:16.871910 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-281db89e-3977-4cb6-928b-5ccecea8974b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:09:16.913498 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 507/1016] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:09:16 2022] POST /identity/v3/auth/tokens => generated 1873 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 09:09:16.929020 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:16.947476 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 510/1017] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:09:16 2022] POST /identity/v3/auth/tokens => generated 1873 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 09:09:17.179293 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d1152e-c9de-49d1-ae6d-8e168162dd88 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:17.217179 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281db89e-3977-4cb6-928b-5ccecea8974b None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:17.242177 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:17.289320 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d1152e-c9de-49d1-ae6d-8e168162dd88 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:17.324515 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281db89e-3977-4cb6-928b-5ccecea8974b None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:17.330176 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279d518c-815c-4888-83b1-d4266c9d9ba1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:17.331272 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279d518c-815c-4888-83b1-d4266c9d9ba1 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:17.331847 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279d518c-815c-4888-83b1-d4266c9d9ba1 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:17.332341 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279d518c-815c-4888-83b1-d4266c9d9ba1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:17.334326 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279d518c-815c-4888-83b1-d4266c9d9ba1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:17.337801 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279d518c-815c-4888-83b1-d4266c9d9ba1 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:17.346076 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 508/1018] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 24 09:09:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:09:17.346894 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279d518c-815c-4888-83b1-d4266c9d9ba1 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:17.376161 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4be4ee-d50c-4119-89d9-17e66e93d8ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:17.376767 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4be4ee-d50c-4119-89d9-17e66e93d8ea None admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:17.377081 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4be4ee-d50c-4119-89d9-17e66e93d8ea None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:17.377307 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4be4ee-d50c-4119-89d9-17e66e93d8ea None admin] PATH_INFO: `/v3/services` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:17.377947 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4be4ee-d50c-4119-89d9-17e66e93d8ea None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:17.379289 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4be4ee-d50c-4119-89d9-17e66e93d8ea None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:17.451334 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279d518c-815c-4888-83b1-d4266c9d9ba1 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:17.475304 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 511/1019] 10.222.0.16 () {60 vars in 1141 bytes} [Wed Aug 24 09:09:17 2022] POST /identity/v3/services => generated 242 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 09:09:17.478338 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:17.495362 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c496094-aa6b-42a2-8e3c-d430cd743000 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:17.496155 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c496094-aa6b-42a2-8e3c-d430cd743000 None admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:17.496371 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c496094-aa6b-42a2-8e3c-d430cd743000 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:17.496588 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c496094-aa6b-42a2-8e3c-d430cd743000 None admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:17.497252 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c496094-aa6b-42a2-8e3c-d430cd743000 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:17.498780 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c496094-aa6b-42a2-8e3c-d430cd743000 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:17.603617 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 509/1020] 10.222.0.16 () {60 vars in 1141 bytes} [Wed Aug 24 09:09:17 2022] POST /identity/v3/projects => generated 311 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 09:09:18.693947 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc38ff4-24ff-4447-8f1c-2af20eddb8ee None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:18.694460 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc38ff4-24ff-4447-8f1c-2af20eddb8ee None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:18.694460 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc38ff4-24ff-4447-8f1c-2af20eddb8ee None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:18.696550 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 512/1021] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:09:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:09:18.702363 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2b17aa-3670-427c-a1ec-448d5cfa878b None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:18.702834 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2b17aa-3670-427c-a1ec-448d5cfa878b None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:18.702931 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2b17aa-3670-427c-a1ec-448d5cfa878b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:18.721594 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-ce2b17aa-3670-427c-a1ec-448d5cfa878b None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:09:18.722773 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-ce2b17aa-3670-427c-a1ec-448d5cfa878b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:09:18.790473 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 510/1022] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:09:18 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 09:09:19.062974 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc38ff4-24ff-4447-8f1c-2af20eddb8ee None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:19.111514 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:19.167952 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc38ff4-24ff-4447-8f1c-2af20eddb8ee None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:19.202899 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c61bcc7-0a87-49a9-b0be-1f326e3d96eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:19.203822 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c61bcc7-0a87-49a9-b0be-1f326e3d96eb None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:19.204105 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c61bcc7-0a87-49a9-b0be-1f326e3d96eb None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:19.204374 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c61bcc7-0a87-49a9-b0be-1f326e3d96eb None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:19.215530 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c61bcc7-0a87-49a9-b0be-1f326e3d96eb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:09:19.220310 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-7c61bcc7-0a87-49a9-b0be-1f326e3d96eb None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 09:09:19.220310 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:09:19.220310 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:09:19.220310 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:09:19.220310 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:09:19.220310 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:09:19.220310 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:09:19.220310 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:09:19.220310 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:09:19.220310 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:09:19.220310 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:09:19.220310 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:09:19.220310 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:09:19.220310 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:09:19.220310 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:09:19.220310 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:09:19.221251 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:09:19.221251 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:09:19.221251 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:09:19.221251 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:09:19.221251 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:09:19.221251 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:09:19.221251 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:09:19.221251 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:09:19.221251 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:09:19.221251 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:09:19.221251 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:09:19.221251 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Aug 24 09:09:19.221251 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:09:19.221251 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:09:19.221251 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:09:19.221251 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:09:19.224570 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:09:19.224570 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:09:19.224570 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:09:19.224570 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:09:19.224570 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:09:19.224570 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:09:19.224570 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:09:19.224570 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:09:19.224570 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:09:19.224570 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:09:19.224570 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 09:09:19.224570 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Aug 24 09:09:19.224570 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 513/1023] 10.222.0.16 () {56 vars in 1120 bytes} [Wed Aug 24 09:09:19 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:09:19.227456 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2b17aa-3670-427c-a1ec-448d5cfa878b None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:19.335547 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2b17aa-3670-427c-a1ec-448d5cfa878b None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:19.338738 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:19.371228 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863453a1-31a8-475b-9c5c-91ee6ba05283 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:19.372518 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863453a1-31a8-475b-9c5c-91ee6ba05283 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:19.373490 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863453a1-31a8-475b-9c5c-91ee6ba05283 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:19.373854 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863453a1-31a8-475b-9c5c-91ee6ba05283 None admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:19.374451 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863453a1-31a8-475b-9c5c-91ee6ba05283 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:09:19.375021 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863453a1-31a8-475b-9c5c-91ee6ba05283 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:19.376464 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863453a1-31a8-475b-9c5c-91ee6ba05283 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:19.387744 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 511/1024] 10.222.0.16 () {56 vars in 1115 bytes} [Wed Aug 24 09:09:19 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:09:19.391308 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c61bcc7-0a87-49a9-b0be-1f326e3d96eb None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:19.479052 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7836db-8d14-4c7e-88da-391a63e1bf10 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:19.479345 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7836db-8d14-4c7e-88da-391a63e1bf10 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:19.479598 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7836db-8d14-4c7e-88da-391a63e1bf10 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:19.481081 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 512/1025] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:09:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 09:09:19.485973 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a2ce6e-c111-4ccf-b42d-4b18029fe1d1 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:19.486256 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a2ce6e-c111-4ccf-b42d-4b18029fe1d1 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:19.486531 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a2ce6e-c111-4ccf-b42d-4b18029fe1d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:19.498998 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c61bcc7-0a87-49a9-b0be-1f326e3d96eb None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:19.501417 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:19.505092 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-43a2ce6e-c111-4ccf-b42d-4b18029fe1d1 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:09:19.506264 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-43a2ce6e-c111-4ccf-b42d-4b18029fe1d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:09:19.534843 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89128f19-63cf-41c8-81b2-f62465a71e0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:19.535484 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89128f19-63cf-41c8-81b2-f62465a71e0e None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:19.536218 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89128f19-63cf-41c8-81b2-f62465a71e0e None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:19.536797 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89128f19-63cf-41c8-81b2-f62465a71e0e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:19.548647 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89128f19-63cf-41c8-81b2-f62465a71e0e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:19.550086 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89128f19-63cf-41c8-81b2-f62465a71e0e None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:19.564283 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-89128f19-63cf-41c8-81b2-f62465a71e0e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:09:19.564283 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:09:19.564283 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:09:19.564283 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:09:19.564283 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:09:19.564283 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:09:19.564283 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:09:19.564283 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:09:19.564283 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:09:19.564283 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:09:19.564283 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:09:19.564283 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:09:19.564283 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:09:19.564283 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:09:19.564283 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:09:19.564283 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:09:19.565252 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:09:19.565252 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:09:19.565252 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:09:19.565252 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:09:19.565252 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:09:19.565252 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:09:19.565252 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:09:19.565252 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Aug 24 09:09:19.565252 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:09:19.565252 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:09:19.565252 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:09:19.565252 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:09:19.565252 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:09:19.565252 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:09:19.565252 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:09:19.565252 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:09:19.565252 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:09:19.566193 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:09:19.566193 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:09:19.566193 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Aug 24 09:09:19.566193 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 514/1026] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 24 09:09:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 09:09:19.566650 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c6867a-2387-403b-abec-16eb9321af54 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:19.566954 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c6867a-2387-403b-abec-16eb9321af54 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:19.567169 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c6867a-2387-403b-abec-16eb9321af54 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:19.568383 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 515/1027] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:09:19 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:09:19.571757 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c6867a-2387-403b-abec-16eb9321af54 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:19.579619 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 513/1028] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:09:19 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 09:09:19.580716 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7f3d3d-b277-4bbf-8515-d8fc464e37b3 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:19.581056 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7f3d3d-b277-4bbf-8515-d8fc464e37b3 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:19.581401 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7f3d3d-b277-4bbf-8515-d8fc464e37b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:19.600230 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-ff7f3d3d-b277-4bbf-8515-d8fc464e37b3 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:09:19.601346 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-ff7f3d3d-b277-4bbf-8515-d8fc464e37b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:09:19.672705 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 514/1029] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:09:19 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 09:09:19.674941 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c6867a-2387-403b-abec-16eb9321af54 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:19.708885 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b6367e-587e-46d5-bebd-a808173f6dc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:19.709586 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b6367e-587e-46d5-bebd-a808173f6dc1 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:19.709824 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b6367e-587e-46d5-bebd-a808173f6dc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:19.710096 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b6367e-587e-46d5-bebd-a808173f6dc1 None admin] PATH_INFO: `/v3/domains` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:19.710780 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b6367e-587e-46d5-bebd-a808173f6dc1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:09:19.711082 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b6367e-587e-46d5-bebd-a808173f6dc1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:19.712605 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b6367e-587e-46d5-bebd-a808173f6dc1 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:19.724632 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 516/1030] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Aug 24 09:09:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:09:19.730557 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7f3d3d-b277-4bbf-8515-d8fc464e37b3 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:19.758357 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:19.829780 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7f3d3d-b277-4bbf-8515-d8fc464e37b3 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:19.851564 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b6367e-587e-46d5-bebd-a808173f6dc1 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:19.869342 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad793029-9743-411d-a09f-862bf1d741bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:19.870282 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad793029-9743-411d-a09f-862bf1d741bc None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:19.870769 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad793029-9743-411d-a09f-862bf1d741bc None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:19.871241 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad793029-9743-411d-a09f-862bf1d741bc None admin] PATH_INFO: `/v3/users/nova` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:19.871496 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:19.879102 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad793029-9743-411d-a09f-862bf1d741bc None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:19.880747 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad793029-9743-411d-a09f-862bf1d741bc None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:19.892453 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-ad793029-9743-411d-a09f-862bf1d741bc None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 09:09:19.892453 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:09:19.892453 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:09:19.892453 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:09:19.892453 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:09:19.892453 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:09:19.892453 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:09:19.892453 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:09:19.892453 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:09:19.892453 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:09:19.892453 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:09:19.892453 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:09:19.892453 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 09:09:19.892453 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 09:09:19.892453 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 09:09:19.892453 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 09:09:19.893640 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:09:19.893640 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:09:19.893640 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 09:09:19.893640 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:09:19.893640 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 09:09:19.893640 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:09:19.893640 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:09:19.893640 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:09:19.893640 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:09:19.893640 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:09:19.893640 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:09:19.893640 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 24 09:09:19.893640 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:09:19.893640 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:09:19.893640 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:09:19.893640 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:09:19.893640 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:09:19.893640 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:09:19.894826 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:09:19.894826 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:09:19.894826 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 09:09:19.894826 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 09:09:19.894826 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 09:09:19.894826 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 09:09:19.894826 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 09:09:19.894826 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 09:09:19.894826 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 09:09:19.894826 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 24 09:09:19.896986 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 515/1031] 10.222.0.16 () {56 vars in 1093 bytes} [Wed Aug 24 09:09:19 2022] GET /identity/v3/users/nova => generated 82 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 09:09:19.902859 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad793029-9743-411d-a09f-862bf1d741bc None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:19.959256 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b6367e-587e-46d5-bebd-a808173f6dc1 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:19.990484 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:19.994212 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bceaff5c-d4d5-40ad-8685-013aa9887f2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:19.994836 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceaff5c-d4d5-40ad-8685-013aa9887f2e None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:19.995108 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceaff5c-d4d5-40ad-8685-013aa9887f2e None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:19.995382 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceaff5c-d4d5-40ad-8685-013aa9887f2e None admin] PATH_INFO: `/v3/services/image` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:19.996426 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bceaff5c-d4d5-40ad-8685-013aa9887f2e None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:19.997937 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bceaff5c-d4d5-40ad-8685-013aa9887f2e None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:20.005188 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad793029-9743-411d-a09f-862bf1d741bc None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:20.006400 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-bceaff5c-d4d5-40ad-8685-013aa9887f2e None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 09:09:20.006400 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:09:20.006400 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 09:09:20.006400 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 09:09:20.006400 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 09:09:20.006400 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 09:09:20.006400 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 09:09:20.006400 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:09:20.006400 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 09:09:20.006400 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Aug 24 09:09:20.006400 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 09:09:20.006400 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Aug 24 09:09:20.006400 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:09:20.006400 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:09:20.006400 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:09:20.006400 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:09:20.006400 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:09:20.009140 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:09:20.009140 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:09:20.009140 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:09:20.009140 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:09:20.009140 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:09:20.009140 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:09:20.009140 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 09:09:20.009140 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 09:09:20.009140 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 09:09:20.009140 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 09:09:20.009140 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:09:20.009140 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:09:20.009140 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:09:20.009140 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:09:20.009140 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:09:20.009140 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:09:20.010018 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:09:20.010018 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Aug 24 09:09:20.010018 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:09:20.010018 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:09:20.010018 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:09:20.010018 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:09:20.010018 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:09:20.010018 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:09:20.010018 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:09:20.010018 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:09:20.010018 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 09:09:20.010018 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:09:20.010018 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 09:09:20.010018 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Aug 24 09:09:20.010018 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 517/1032] 10.222.0.16 () {56 vars in 1105 bytes} [Wed Aug 24 09:09:19 2022] GET /identity/v3/services/image => generated 86 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 09:09:20.010834 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bceaff5c-d4d5-40ad-8685-013aa9887f2e None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:20.041434 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44752e41-3f58-4a94-84a6-ca502f4f0c56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:20.042043 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44752e41-3f58-4a94-84a6-ca502f4f0c56 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:20.042321 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44752e41-3f58-4a94-84a6-ca502f4f0c56 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:20.042597 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44752e41-3f58-4a94-84a6-ca502f4f0c56 None admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:20.043314 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44752e41-3f58-4a94-84a6-ca502f4f0c56 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:09:20.043640 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44752e41-3f58-4a94-84a6-ca502f4f0c56 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:20.045047 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44752e41-3f58-4a94-84a6-ca502f4f0c56 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:20.061506 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 516/1033] 10.222.0.16 () {56 vars in 1133 bytes} [Wed Aug 24 09:09:19 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:09:20.062298 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44752e41-3f58-4a94-84a6-ca502f4f0c56 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:20.093114 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:20.113113 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bceaff5c-d4d5-40ad-8685-013aa9887f2e None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:20.146767 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a78d054-6466-4a8f-8615-11fc90119159 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:20.150661 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a78d054-6466-4a8f-8615-11fc90119159 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:20.150661 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a78d054-6466-4a8f-8615-11fc90119159 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:20.150661 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a78d054-6466-4a8f-8615-11fc90119159 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:20.150661 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a78d054-6466-4a8f-8615-11fc90119159 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:20.151133 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a78d054-6466-4a8f-8615-11fc90119159 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:20.152734 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 518/1034] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 24 09:09:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:09:20.153493 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a78d054-6466-4a8f-8615-11fc90119159 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:20.165259 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44752e41-3f58-4a94-84a6-ca502f4f0c56 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:20.199216 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af82a137-96bd-4831-b7e2-b2b6febec692 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:20.199853 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af82a137-96bd-4831-b7e2-b2b6febec692 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:20.200128 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af82a137-96bd-4831-b7e2-b2b6febec692 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:20.200398 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af82a137-96bd-4831-b7e2-b2b6febec692 None admin] PATH_INFO: `/v3/services` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:20.201019 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af82a137-96bd-4831-b7e2-b2b6febec692 None admin] RBAC: Adding query filter params (name=image) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:09:20.201321 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af82a137-96bd-4831-b7e2-b2b6febec692 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:20.202721 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af82a137-96bd-4831-b7e2-b2b6febec692 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:20.207667 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 517/1035] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 24 09:09:20 2022] GET /identity/v3/services?name=image => generated 122 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:09:20.208418 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af82a137-96bd-4831-b7e2-b2b6febec692 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:20.222083 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:20.253650 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a78d054-6466-4a8f-8615-11fc90119159 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:20.287137 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9e1363-bcb1-4beb-95b6-a1b6cb2345f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:20.287871 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9e1363-bcb1-4beb-95b6-a1b6cb2345f0 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:20.288151 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9e1363-bcb1-4beb-95b6-a1b6cb2345f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:20.288431 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9e1363-bcb1-4beb-95b6-a1b6cb2345f0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:20.307912 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:20.311067 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af82a137-96bd-4831-b7e2-b2b6febec692 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:20.345736 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73aa6e5-3493-42c1-acd6-b042443dad91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:20.346341 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73aa6e5-3493-42c1-acd6-b042443dad91 None admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:20.346619 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73aa6e5-3493-42c1-acd6-b042443dad91 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:20.346895 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73aa6e5-3493-42c1-acd6-b042443dad91 None admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:20.347530 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73aa6e5-3493-42c1-acd6-b042443dad91 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:20.348999 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73aa6e5-3493-42c1-acd6-b042443dad91 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:20.358965 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9e1363-bcb1-4beb-95b6-a1b6cb2345f0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:20.361172 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9e1363-bcb1-4beb-95b6-a1b6cb2345f0 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:20.460433 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 519/1036] 10.222.0.16 () {58 vars in 1282 bytes} [Wed Aug 24 09:09:20 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 09:09:20.461208 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9e1363-bcb1-4beb-95b6-a1b6cb2345f0 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:20.470930 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 518/1037] 10.222.0.16 () {60 vars in 1132 bytes} [Wed Aug 24 09:09:20 2022] POST /identity/v3/users => generated 285 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:09:20.472073 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73aa6e5-3493-42c1-acd6-b042443dad91 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:20.530905 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:20.544147 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:20.585786 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73aa6e5-3493-42c1-acd6-b042443dad91 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:20.596863 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9e1363-bcb1-4beb-95b6-a1b6cb2345f0 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:20.630052 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81922743-495b-49db-956a-2e04a6d07d03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:20.630833 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81922743-495b-49db-956a-2e04a6d07d03 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:20.631213 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81922743-495b-49db-956a-2e04a6d07d03 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:20.631531 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81922743-495b-49db-956a-2e04a6d07d03 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:20.641164 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e801add2-b4da-4f6f-a6f6-1cb20721aa7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:20.642025 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e801add2-b4da-4f6f-a6f6-1cb20721aa7e None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:20.642306 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e801add2-b4da-4f6f-a6f6-1cb20721aa7e None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:20.642579 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e801add2-b4da-4f6f-a6f6-1cb20721aa7e None admin] PATH_INFO: `/v3/services` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:20.643211 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e801add2-b4da-4f6f-a6f6-1cb20721aa7e None admin] RBAC: Adding query filter params (type=image) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:09:20.643514 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e801add2-b4da-4f6f-a6f6-1cb20721aa7e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:20.643840 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81922743-495b-49db-956a-2e04a6d07d03 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:20.644919 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e801add2-b4da-4f6f-a6f6-1cb20721aa7e None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:20.645286 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81922743-495b-49db-956a-2e04a6d07d03 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:20.650546 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 520/1038] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 24 09:09:20 2022] GET /identity/v3/services?type=image => generated 350 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:09:20.657036 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e801add2-b4da-4f6f-a6f6-1cb20721aa7e None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:20.665189 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-81922743-495b-49db-956a-2e04a6d07d03 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:09:20.665189 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:09:20.665189 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:09:20.665189 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:09:20.665189 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:09:20.665189 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:09:20.665189 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:09:20.665189 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:09:20.665189 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:09:20.665189 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:09:20.665189 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:09:20.665189 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:09:20.665189 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:09:20.665189 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:09:20.665189 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:09:20.665189 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:09:20.666138 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:09:20.666138 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:09:20.666138 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:09:20.666138 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:09:20.666138 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:09:20.666138 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:09:20.666138 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:09:20.666138 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 24 09:09:20.666138 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:09:20.666138 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:09:20.666138 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:09:20.666138 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:09:20.666138 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:09:20.666138 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:09:20.666138 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:09:20.666138 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:09:20.666138 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:09:20.667047 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:09:20.667047 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:09:20.667047 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 24 09:09:20.667047 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 519/1039] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 24 09:09:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:09:20.672065 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81922743-495b-49db-956a-2e04a6d07d03 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:20.772891 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e801add2-b4da-4f6f-a6f6-1cb20721aa7e None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:20.777956 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81922743-495b-49db-956a-2e04a6d07d03 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:20.788919 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:20.795220 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:20.810836 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d0b9db-a50e-4d50-8e1d-dc1ef6e61a88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:20.811454 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d0b9db-a50e-4d50-8e1d-dc1ef6e61a88 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:20.811769 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d0b9db-a50e-4d50-8e1d-dc1ef6e61a88 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:20.812081 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d0b9db-a50e-4d50-8e1d-dc1ef6e61a88 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:20.812746 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d0b9db-a50e-4d50-8e1d-dc1ef6e61a88 None admin] RBAC: Adding query filter params (interface=public, service_id=c2c830c9d6b646f89e50503af1369625, region_id=RegionOne) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:09:20.813057 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d0b9db-a50e-4d50-8e1d-dc1ef6e61a88 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:20.813462 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe163ffb-3be6-48b7-85a7-9f037b332c20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:20.814073 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe163ffb-3be6-48b7-85a7-9f037b332c20 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:20.814350 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe163ffb-3be6-48b7-85a7-9f037b332c20 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:20.814446 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d0b9db-a50e-4d50-8e1d-dc1ef6e61a88 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:20.814620 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe163ffb-3be6-48b7-85a7-9f037b332c20 None admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:20.815259 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe163ffb-3be6-48b7-85a7-9f037b332c20 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:09:20.815559 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe163ffb-3be6-48b7-85a7-9f037b332c20 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:20.816935 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe163ffb-3be6-48b7-85a7-9f037b332c20 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:20.824128 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 521/1040] 10.222.0.16 () {56 vars in 1251 bytes} [Wed Aug 24 09:09:20 2022] GET /identity/v3/endpoints?service_id=c2c830c9d6b646f89e50503af1369625&interface=public®ion_id=RegionOne => generated 194 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:09:20.825484 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d0b9db-a50e-4d50-8e1d-dc1ef6e61a88 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:20.829498 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 520/1041] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Aug 24 09:09:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:09:20.835319 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe163ffb-3be6-48b7-85a7-9f037b332c20 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:20.933554 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d0b9db-a50e-4d50-8e1d-dc1ef6e61a88 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:20.941409 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe163ffb-3be6-48b7-85a7-9f037b332c20 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:20.972367 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ac5410-98a0-48c2-be3b-64fcf68f294f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:20.974197 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ac5410-98a0-48c2-be3b-64fcf68f294f None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:20.974197 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ac5410-98a0-48c2-be3b-64fcf68f294f None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:20.974197 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ac5410-98a0-48c2-be3b-64fcf68f294f None admin] PATH_INFO: `/v3/services` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:20.974197 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ac5410-98a0-48c2-be3b-64fcf68f294f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:20.975488 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ac5410-98a0-48c2-be3b-64fcf68f294f None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:20.979877 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a37f2a7-a657-4d27-8486-7ec8ade3f714 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:20.980510 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a37f2a7-a657-4d27-8486-7ec8ade3f714 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:20.980786 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a37f2a7-a657-4d27-8486-7ec8ade3f714 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:20.981056 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a37f2a7-a657-4d27-8486-7ec8ade3f714 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:20.991857 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 522/1042] 10.222.0.16 () {56 vars in 1087 bytes} [Wed Aug 24 09:09:20 2022] GET /identity/v3/services => generated 1957 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:09:20.998311 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a37f2a7-a657-4d27-8486-7ec8ade3f714 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:20.999954 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a37f2a7-a657-4d27-8486-7ec8ade3f714 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:21.013241 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-8a37f2a7-a657-4d27-8486-7ec8ade3f714 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:09:21.013241 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:09:21.013241 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:09:21.013241 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:09:21.013241 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:09:21.013241 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:09:21.013241 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:09:21.013241 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:09:21.013241 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:09:21.013241 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:09:21.013241 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:09:21.013241 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:09:21.013241 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 09:09:21.013241 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 09:09:21.013241 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 09:09:21.014247 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 09:09:21.014247 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:09:21.014247 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:09:21.014247 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:09:21.014247 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:09:21.014247 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:09:21.014247 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:09:21.014247 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:09:21.014247 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 24 09:09:21.014247 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:09:21.014247 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:09:21.014247 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:09:21.014247 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:09:21.014247 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:09:21.014247 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:09:21.014247 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:09:21.014247 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:09:21.015240 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 09:09:21.015240 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 09:09:21.015240 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 09:09:21.015240 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 09:09:21.015240 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 09:09:21.015240 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 09:09:21.015240 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:09:21.015240 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 24 09:09:21.019441 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 521/1043] 10.222.0.16 () {56 vars in 1111 bytes} [Wed Aug 24 09:09:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:09:21.020750 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ac5410-98a0-48c2-be3b-64fcf68f294f None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:21.068570 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:21.132559 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ac5410-98a0-48c2-be3b-64fcf68f294f None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:21.169710 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb8ab7a-1cfa-4292-a2fc-ff46f81d6910 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:21.170371 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb8ab7a-1cfa-4292-a2fc-ff46f81d6910 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:21.170654 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb8ab7a-1cfa-4292-a2fc-ff46f81d6910 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:21.170926 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb8ab7a-1cfa-4292-a2fc-ff46f81d6910 None admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:21.171639 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb8ab7a-1cfa-4292-a2fc-ff46f81d6910 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:09:21.171949 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb8ab7a-1cfa-4292-a2fc-ff46f81d6910 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:21.173377 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb8ab7a-1cfa-4292-a2fc-ff46f81d6910 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:21.185858 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 523/1044] 10.222.0.16 () {56 vars in 1148 bytes} [Wed Aug 24 09:09:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:09:21.191663 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a37f2a7-a657-4d27-8486-7ec8ade3f714 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:21.229488 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:21.293913 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a37f2a7-a657-4d27-8486-7ec8ade3f714 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:21.332888 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92074624-9825-4d17-8b98-f7be1d0101d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:21.333435 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92074624-9825-4d17-8b98-f7be1d0101d3 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:21.333716 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92074624-9825-4d17-8b98-f7be1d0101d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:21.333989 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92074624-9825-4d17-8b98-f7be1d0101d3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:21.334723 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92074624-9825-4d17-8b98-f7be1d0101d3 None admin] RBAC: Adding query filter params (role.id=716e298774d64bc89a265236b7b9db7f, scope.project.id=c467e88e2fb0491bacefcf6d2ce326d9, user.id=1f055f7dc51d43bda9b315ca67f11d58) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:09:21.335034 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92074624-9825-4d17-8b98-f7be1d0101d3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:21.336546 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92074624-9825-4d17-8b98-f7be1d0101d3 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:21.351628 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 522/1045] 10.222.0.16 () {56 vars in 1374 bytes} [Wed Aug 24 09:09:21 2022] GET /identity/v3/role_assignments?user.id=1f055f7dc51d43bda9b315ca67f11d58&scope.project.id=c467e88e2fb0491bacefcf6d2ce326d9&role.id=716e298774d64bc89a265236b7b9db7f => generated 259 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:09:22.438357 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e21232a-e458-452b-8f8d-28ba1385b2be None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:22.438891 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e21232a-e458-452b-8f8d-28ba1385b2be None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:22.438891 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e21232a-e458-452b-8f8d-28ba1385b2be None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:22.440300 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 524/1046] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:09:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:09:22.445599 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2563d655-f714-4405-93b7-6425c01b8ce4 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:22.446015 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2563d655-f714-4405-93b7-6425c01b8ce4 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:22.450321 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2563d655-f714-4405-93b7-6425c01b8ce4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:22.468553 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-2563d655-f714-4405-93b7-6425c01b8ce4 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:09:22.469719 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-2563d655-f714-4405-93b7-6425c01b8ce4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:09:22.537206 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 523/1047] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:09:22 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 09:09:22.805091 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e21232a-e458-452b-8f8d-28ba1385b2be None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:22.901640 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20064a38-1d76-4c9c-98fb-ac1874e98a05 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:22.902028 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20064a38-1d76-4c9c-98fb-ac1874e98a05 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:22.902362 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20064a38-1d76-4c9c-98fb-ac1874e98a05 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:22.903877 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 524/1048] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:09:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:09:22.905198 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:22.909106 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f4780c-69ea-48b1-9059-ff81b2ee7c75 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:22.909387 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f4780c-69ea-48b1-9059-ff81b2ee7c75 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:22.909654 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f4780c-69ea-48b1-9059-ff81b2ee7c75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:22.909940 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e21232a-e458-452b-8f8d-28ba1385b2be None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:22.928766 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-41f4780c-69ea-48b1-9059-ff81b2ee7c75 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:09:22.931745 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-41f4780c-69ea-48b1-9059-ff81b2ee7c75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:09:22.945426 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2c9f0f-fc57-4848-9a5b-33705a3ddff0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:22.946066 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2c9f0f-fc57-4848-9a5b-33705a3ddff0 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:22.946348 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2c9f0f-fc57-4848-9a5b-33705a3ddff0 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:22.946636 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2c9f0f-fc57-4848-9a5b-33705a3ddff0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:22.957134 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:22.958474 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2c9f0f-fc57-4848-9a5b-33705a3ddff0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:09:22.963373 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-5c2c9f0f-fc57-4848-9a5b-33705a3ddff0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:09:22.963373 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:09:22.963373 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:09:22.963373 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:09:22.963373 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:09:22.963373 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:09:22.963373 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:09:22.963373 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:09:22.963373 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:09:22.963373 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:09:22.963373 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:09:22.963373 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:09:22.963373 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:09:22.963373 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:09:22.963373 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:09:22.963373 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:09:22.968534 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:09:22.968534 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:09:22.968534 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:09:22.968534 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:09:22.968534 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:09:22.968534 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:09:22.968534 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:09:22.968534 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:09:22.968534 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:09:22.968534 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:09:22.968534 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:09:22.968534 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Aug 24 09:09:22.968534 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:09:22.968534 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:09:22.968534 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:09:22.968534 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:09:22.969390 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:09:22.969390 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:09:22.969390 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:09:22.969390 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:09:22.969390 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:09:22.969390 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:09:22.969390 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:09:22.969390 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:09:22.969390 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:09:22.969390 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:09:22.969390 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:09:22.969390 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Aug 24 09:09:22.969390 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 525/1049] 10.222.0.16 () {56 vars in 1096 bytes} [Wed Aug 24 09:09:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:09:22.970688 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2c9f0f-fc57-4848-9a5b-33705a3ddff0 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:23.009658 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 525/1050] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:09:22 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 09:09:23.075035 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2c9f0f-fc57-4848-9a5b-33705a3ddff0 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:23.109977 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a168e39f-72eb-402e-ae78-5520b2ccedd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:23.110580 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a168e39f-72eb-402e-ae78-5520b2ccedd8 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:23.110856 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a168e39f-72eb-402e-ae78-5520b2ccedd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:23.111124 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a168e39f-72eb-402e-ae78-5520b2ccedd8 None admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:23.111797 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a168e39f-72eb-402e-ae78-5520b2ccedd8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:09:23.112098 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a168e39f-72eb-402e-ae78-5520b2ccedd8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:23.113444 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a168e39f-72eb-402e-ae78-5520b2ccedd8 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:23.122255 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 526/1051] 10.222.0.16 () {56 vars in 1099 bytes} [Wed Aug 24 09:09:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:09:23.127944 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f4780c-69ea-48b1-9059-ff81b2ee7c75 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:23.223217 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6309d53e-5269-4c57-9be7-d44080b07a8c None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:23.223507 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6309d53e-5269-4c57-9be7-d44080b07a8c None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:23.223826 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6309d53e-5269-4c57-9be7-d44080b07a8c None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:23.225162 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 527/1052] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:09:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:09:23.230142 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e83154-b24a-4694-a175-2f6be84c479b None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:23.230428 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e83154-b24a-4694-a175-2f6be84c479b None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:23.230703 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e83154-b24a-4694-a175-2f6be84c479b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:23.231089 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f4780c-69ea-48b1-9059-ff81b2ee7c75 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:23.248288 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-25e83154-b24a-4694-a175-2f6be84c479b None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:09:23.249479 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-25e83154-b24a-4694-a175-2f6be84c479b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:09:23.266749 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f39df2-2b0b-4779-9cb1-2a3b48040f22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:23.267390 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f39df2-2b0b-4779-9cb1-2a3b48040f22 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:23.267504 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:23.267695 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f39df2-2b0b-4779-9cb1-2a3b48040f22 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:23.267976 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f39df2-2b0b-4779-9cb1-2a3b48040f22 None admin] PATH_INFO: `/v3/users/b3dab487cd8a43ac9a7760c8df3a2b7c` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:23.283104 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f39df2-2b0b-4779-9cb1-2a3b48040f22 None admin] RBAC: Authorizing `identity:get_user(user_id=b3dab487cd8a43ac9a7760c8df3a2b7c)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:23.284582 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f39df2-2b0b-4779-9cb1-2a3b48040f22 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:23.290913 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 526/1053] 10.222.0.16 () {56 vars in 1177 bytes} [Wed Aug 24 09:09:23 2022] GET /identity/v3/users/b3dab487cd8a43ac9a7760c8df3a2b7c => generated 285 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:09:23.290913 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f39df2-2b0b-4779-9cb1-2a3b48040f22 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:23.309320 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:23.321593 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 528/1054] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:09:23 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 09:09:23.322442 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e83154-b24a-4694-a175-2f6be84c479b None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:23.406745 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f39df2-2b0b-4779-9cb1-2a3b48040f22 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:23.432263 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e83154-b24a-4694-a175-2f6be84c479b None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:23.446000 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfecd82-cf20-411f-bc24-1e4e2c976da6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:23.446634 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfecd82-cf20-411f-bc24-1e4e2c976da6 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:23.446922 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfecd82-cf20-411f-bc24-1e4e2c976da6 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:23.447196 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfecd82-cf20-411f-bc24-1e4e2c976da6 None admin] PATH_INFO: `/v3/services/image` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:23.447816 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfecd82-cf20-411f-bc24-1e4e2c976da6 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:23.449174 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfecd82-cf20-411f-bc24-1e4e2c976da6 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:23.458279 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-3bfecd82-cf20-411f-bc24-1e4e2c976da6 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 09:09:23.458279 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:09:23.458279 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 09:09:23.458279 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 09:09:23.458279 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 09:09:23.458279 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 09:09:23.458279 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 09:09:23.458279 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:09:23.458279 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 09:09:23.458279 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 24 09:09:23.458279 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 09:09:23.458279 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 24 09:09:23.458279 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:09:23.458279 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:09:23.458279 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:09:23.458279 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:09:23.458279 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:09:23.459317 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:09:23.459317 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:09:23.459317 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:09:23.459317 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:09:23.459317 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:09:23.459317 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:09:23.459317 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 09:09:23.459317 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 09:09:23.459317 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 09:09:23.459317 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 09:09:23.459317 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:09:23.459317 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:09:23.459317 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:09:23.459317 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:09:23.459317 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:09:23.459317 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:09:23.460219 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:09:23.460219 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 24 09:09:23.460219 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:09:23.460219 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:09:23.460219 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:09:23.460219 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:09:23.460219 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:09:23.460219 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:09:23.460219 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:09:23.460219 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:09:23.460219 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 09:09:23.460219 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:09:23.460219 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 09:09:23.460219 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 24 09:09:23.460219 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 527/1055] 10.222.0.16 () {56 vars in 1105 bytes} [Wed Aug 24 09:09:23 2022] GET /identity/v3/services/image => generated 86 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:09:23.465667 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfecd82-cf20-411f-bc24-1e4e2c976da6 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:23.470941 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575b78de-6997-4a07-b17b-86b8b1ab90dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:23.471561 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575b78de-6997-4a07-b17b-86b8b1ab90dd None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:23.471874 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575b78de-6997-4a07-b17b-86b8b1ab90dd None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:23.472150 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575b78de-6997-4a07-b17b-86b8b1ab90dd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:23.495552 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:23.545151 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575b78de-6997-4a07-b17b-86b8b1ab90dd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:23.546718 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575b78de-6997-4a07-b17b-86b8b1ab90dd None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:23.551228 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:23.569237 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfecd82-cf20-411f-bc24-1e4e2c976da6 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:23.604988 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c769fbe8-e2f7-42be-8398-225c00d27b4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:23.605595 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c769fbe8-e2f7-42be-8398-225c00d27b4b None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:23.605874 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c769fbe8-e2f7-42be-8398-225c00d27b4b None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:23.606144 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c769fbe8-e2f7-42be-8398-225c00d27b4b None admin] PATH_INFO: `/v3/services` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:23.606772 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c769fbe8-e2f7-42be-8398-225c00d27b4b None admin] RBAC: Adding query filter params (name=image) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:09:23.607073 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c769fbe8-e2f7-42be-8398-225c00d27b4b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:23.608462 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c769fbe8-e2f7-42be-8398-225c00d27b4b None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:23.613265 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 528/1056] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 24 09:09:23 2022] GET /identity/v3/services?name=image => generated 122 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:09:23.614062 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c769fbe8-e2f7-42be-8398-225c00d27b4b None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:23.652420 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 529/1057] 10.222.0.16 () {58 vars in 1282 bytes} [Wed Aug 24 09:09:23 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 09:09:23.653268 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575b78de-6997-4a07-b17b-86b8b1ab90dd None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:23.706638 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:23.714149 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c769fbe8-e2f7-42be-8398-225c00d27b4b None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:23.747634 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19425333-f92a-4183-8ef7-a0e8a5acc996 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:23.748268 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19425333-f92a-4183-8ef7-a0e8a5acc996 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:23.748567 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19425333-f92a-4183-8ef7-a0e8a5acc996 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:23.748814 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19425333-f92a-4183-8ef7-a0e8a5acc996 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:23.752859 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575b78de-6997-4a07-b17b-86b8b1ab90dd None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:23.759885 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19425333-f92a-4183-8ef7-a0e8a5acc996 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:09:23.764584 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-19425333-f92a-4183-8ef7-a0e8a5acc996 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 09:09:23.764584 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:09:23.764584 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:09:23.764584 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:09:23.764584 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:09:23.764584 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:09:23.764584 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:09:23.764584 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:09:23.764584 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:09:23.764584 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:09:23.764584 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:09:23.764584 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:09:23.764584 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:09:23.764584 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:09:23.764584 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:09:23.764584 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:09:23.765722 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:09:23.765722 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:09:23.765722 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:09:23.765722 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:09:23.765722 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:09:23.765722 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:09:23.765722 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:09:23.765722 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:09:23.765722 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:09:23.765722 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:09:23.765722 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:09:23.765722 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 24 09:09:23.765722 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:09:23.765722 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:09:23.765722 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:09:23.765722 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:09:23.766584 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:09:23.766584 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:09:23.766584 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:09:23.766584 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:09:23.766584 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:09:23.766584 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:09:23.766584 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:09:23.766584 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:09:23.766584 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:09:23.766584 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:09:23.766584 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 09:09:23.766584 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 24 09:09:23.766584 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 529/1058] 10.222.0.16 () {56 vars in 1120 bytes} [Wed Aug 24 09:09:23 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:09:23.766584 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19425333-f92a-4183-8ef7-a0e8a5acc996 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:23.776188 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:23.791673 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb70a8b-5355-40cf-8275-5521638cd777 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:23.792224 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb70a8b-5355-40cf-8275-5521638cd777 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:23.792503 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb70a8b-5355-40cf-8275-5521638cd777 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:23.792877 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb70a8b-5355-40cf-8275-5521638cd777 None admin] PATH_INFO: `/v3/services` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:23.793498 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb70a8b-5355-40cf-8275-5521638cd777 None admin] RBAC: Adding query filter params (type=image) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:09:23.793799 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb70a8b-5355-40cf-8275-5521638cd777 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:23.795159 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb70a8b-5355-40cf-8275-5521638cd777 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:23.799900 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 530/1059] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 24 09:09:23 2022] GET /identity/v3/services?type=image => generated 350 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:09:23.800669 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb70a8b-5355-40cf-8275-5521638cd777 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:23.866692 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19425333-f92a-4183-8ef7-a0e8a5acc996 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:23.900513 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb70a8b-5355-40cf-8275-5521638cd777 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:23.900712 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb24b111-4fdd-4d63-85e9-856f26156b9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:23.901339 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb24b111-4fdd-4d63-85e9-856f26156b9f None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:23.901618 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb24b111-4fdd-4d63-85e9-856f26156b9f None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:23.901899 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb24b111-4fdd-4d63-85e9-856f26156b9f None admin] PATH_INFO: `/v3/projects/e8ae36cb945c488682ab38083533ff07` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:23.903013 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb24b111-4fdd-4d63-85e9-856f26156b9f None admin] RBAC: Authorizing `identity:get_project(project_id=e8ae36cb945c488682ab38083533ff07)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:23.904472 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb24b111-4fdd-4d63-85e9-856f26156b9f None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:23.906095 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 530/1060] 10.222.0.16 () {56 vars in 1186 bytes} [Wed Aug 24 09:09:23 2022] GET /identity/v3/projects/e8ae36cb945c488682ab38083533ff07 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:09:23.906833 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb24b111-4fdd-4d63-85e9-856f26156b9f None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:23.918444 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:23.936765 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cea93b-c6d2-48bb-9efc-9ad6600bdae5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:23.937371 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cea93b-c6d2-48bb-9efc-9ad6600bdae5 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:23.937672 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cea93b-c6d2-48bb-9efc-9ad6600bdae5 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:23.937925 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cea93b-c6d2-48bb-9efc-9ad6600bdae5 None admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:23.938582 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cea93b-c6d2-48bb-9efc-9ad6600bdae5 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:09:23.938887 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cea93b-c6d2-48bb-9efc-9ad6600bdae5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:23.940276 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cea93b-c6d2-48bb-9efc-9ad6600bdae5 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:23.947966 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 531/1061] 10.222.0.16 () {56 vars in 1115 bytes} [Wed Aug 24 09:09:23 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:09:23.948751 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cea93b-c6d2-48bb-9efc-9ad6600bdae5 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:23.983337 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:24.012445 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb24b111-4fdd-4d63-85e9-856f26156b9f None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:24.047605 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7756fd89-1a2b-4e5e-a9a6-121316ba3b8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:24.048212 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7756fd89-1a2b-4e5e-a9a6-121316ba3b8c None admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:24.048490 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7756fd89-1a2b-4e5e-a9a6-121316ba3b8c None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:24.048771 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7756fd89-1a2b-4e5e-a9a6-121316ba3b8c None admin] PATH_INFO: `/v3/endpoints` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:24.049432 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7756fd89-1a2b-4e5e-a9a6-121316ba3b8c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:24.050894 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7756fd89-1a2b-4e5e-a9a6-121316ba3b8c None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:24.054341 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cea93b-c6d2-48bb-9efc-9ad6600bdae5 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:24.087541 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa282de6-6b24-4cdd-857a-5bd0a681af71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:24.088166 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa282de6-6b24-4cdd-857a-5bd0a681af71 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:24.088448 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa282de6-6b24-4cdd-857a-5bd0a681af71 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:24.088721 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa282de6-6b24-4cdd-857a-5bd0a681af71 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:24.089439 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa282de6-6b24-4cdd-857a-5bd0a681af71 None admin] RBAC: Adding query filter params (role.id=15f634d4a2d247ddab90c458e5e1c141, scope.project.id=e8ae36cb945c488682ab38083533ff07, user.id=b3dab487cd8a43ac9a7760c8df3a2b7c) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:09:24.089808 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa282de6-6b24-4cdd-857a-5bd0a681af71 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:24.091191 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa282de6-6b24-4cdd-857a-5bd0a681af71 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:24.096477 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 532/1062] 10.222.0.16 () {56 vars in 1374 bytes} [Wed Aug 24 09:09:23 2022] GET /identity/v3/role_assignments?user.id=b3dab487cd8a43ac9a7760c8df3a2b7c&scope.project.id=e8ae36cb945c488682ab38083533ff07&role.id=15f634d4a2d247ddab90c458e5e1c141 => generated 259 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:09:24.097253 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa282de6-6b24-4cdd-857a-5bd0a681af71 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:24.126555 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 531/1063] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 24 09:09:23 2022] POST /identity/v3/endpoints => generated 327 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:09:24.166937 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:24.235827 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa282de6-6b24-4cdd-857a-5bd0a681af71 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:24.287638 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67482199-c519-4133-895f-a040fc78a674 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:24.288286 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67482199-c519-4133-895f-a040fc78a674 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:24.288572 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67482199-c519-4133-895f-a040fc78a674 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:24.288881 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67482199-c519-4133-895f-a040fc78a674 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:24.303940 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67482199-c519-4133-895f-a040fc78a674 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:24.305436 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67482199-c519-4133-895f-a040fc78a674 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:24.320611 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-67482199-c519-4133-895f-a040fc78a674 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:09:24.320611 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:09:24.320611 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:09:24.320611 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:09:24.320611 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:09:24.320611 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:09:24.320611 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:09:24.320611 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:09:24.320611 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:09:24.320611 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:09:24.320611 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:09:24.320611 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:09:24.320611 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:09:24.320611 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:09:24.320611 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:09:24.320611 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:09:24.321598 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:09:24.321598 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:09:24.321598 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:09:24.321598 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:09:24.321598 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:09:24.321598 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:09:24.321598 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:09:24.321598 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Aug 24 09:09:24.321598 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:09:24.321598 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:09:24.321598 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:09:24.321598 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:09:24.321598 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:09:24.321598 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:09:24.321598 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:09:24.321598 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:09:24.321598 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:09:24.322519 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:09:24.322519 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:09:24.322519 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Aug 24 09:09:24.322519 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 533/1064] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 24 09:09:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:09:24.332084 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7756fd89-1a2b-4e5e-a9a6-121316ba3b8c None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:24.436980 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7756fd89-1a2b-4e5e-a9a6-121316ba3b8c None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:24.466237 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:24.473155 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f631d83-c32c-47a6-be07-0f9fb2f9d2f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:24.473947 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f631d83-c32c-47a6-be07-0f9fb2f9d2f0 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:24.479757 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f631d83-c32c-47a6-be07-0f9fb2f9d2f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:24.479757 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f631d83-c32c-47a6-be07-0f9fb2f9d2f0 None admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:24.479757 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f631d83-c32c-47a6-be07-0f9fb2f9d2f0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:09:24.479757 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f631d83-c32c-47a6-be07-0f9fb2f9d2f0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:24.479757 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f631d83-c32c-47a6-be07-0f9fb2f9d2f0 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:24.490313 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 532/1065] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Aug 24 09:09:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:09:24.495853 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67482199-c519-4133-895f-a040fc78a674 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:24.591073 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:24.601260 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67482199-c519-4133-895f-a040fc78a674 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:24.636964 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febe9a17-e67a-41a7-841f-b205ba172282 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:24.637603 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febe9a17-e67a-41a7-841f-b205ba172282 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:24.637884 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febe9a17-e67a-41a7-841f-b205ba172282 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:24.638156 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febe9a17-e67a-41a7-841f-b205ba172282 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:24.646053 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febe9a17-e67a-41a7-841f-b205ba172282 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:24.647500 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febe9a17-e67a-41a7-841f-b205ba172282 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:24.659344 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-febe9a17-e67a-41a7-841f-b205ba172282 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 09:09:24.659344 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:09:24.659344 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:09:24.659344 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:09:24.659344 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:09:24.659344 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:09:24.659344 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:09:24.659344 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:09:24.659344 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:09:24.659344 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:09:24.659344 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:09:24.659344 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:09:24.659344 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 09:09:24.659344 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 09:09:24.659344 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 09:09:24.659344 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 09:09:24.660481 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:09:24.660481 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:09:24.660481 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 09:09:24.660481 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:09:24.660481 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 09:09:24.660481 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:09:24.660481 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:09:24.660481 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:09:24.660481 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:09:24.660481 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:09:24.660481 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:09:24.660481 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Aug 24 09:09:24.660481 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:09:24.660481 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:09:24.660481 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:09:24.660481 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:09:24.660481 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:09:24.660481 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:09:24.661437 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:09:24.661437 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:09:24.661437 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 09:09:24.661437 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 09:09:24.661437 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 09:09:24.661437 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 09:09:24.661437 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 09:09:24.661437 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 09:09:24.661437 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 09:09:24.661437 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Aug 24 09:09:24.661437 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 534/1066] 10.222.0.16 () {56 vars in 1093 bytes} [Wed Aug 24 09:09:24 2022] GET /identity/v3/users/nova => generated 82 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:09:24.666232 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f631d83-c32c-47a6-be07-0f9fb2f9d2f0 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:24.767124 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f631d83-c32c-47a6-be07-0f9fb2f9d2f0 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:24.800817 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22a1ec9-f32a-41b7-96f6-283fc57c8e76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:24.801416 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22a1ec9-f32a-41b7-96f6-283fc57c8e76 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:24.801690 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22a1ec9-f32a-41b7-96f6-283fc57c8e76 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:24.801955 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22a1ec9-f32a-41b7-96f6-283fc57c8e76 None admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:24.802666 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22a1ec9-f32a-41b7-96f6-283fc57c8e76 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:09:24.802968 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22a1ec9-f32a-41b7-96f6-283fc57c8e76 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:24.804396 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22a1ec9-f32a-41b7-96f6-283fc57c8e76 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:24.826043 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 533/1067] 10.222.0.16 () {56 vars in 1133 bytes} [Wed Aug 24 09:09:24 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:09:24.827385 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febe9a17-e67a-41a7-841f-b205ba172282 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:24.930826 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febe9a17-e67a-41a7-841f-b205ba172282 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:24.964335 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c92dbb-9e29-4300-857b-9f43e857a293 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:24.965110 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c92dbb-9e29-4300-857b-9f43e857a293 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:24.965492 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c92dbb-9e29-4300-857b-9f43e857a293 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:24.965866 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c92dbb-9e29-4300-857b-9f43e857a293 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:24.977657 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c92dbb-9e29-4300-857b-9f43e857a293 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:24.979294 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c92dbb-9e29-4300-857b-9f43e857a293 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:24.993608 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-97c92dbb-9e29-4300-857b-9f43e857a293 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:09:24.993608 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:09:24.993608 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:09:24.993608 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:09:24.993608 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:09:24.993608 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:09:24.993608 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:09:24.993608 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:09:24.993608 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:09:24.993608 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:09:24.993608 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:09:24.993608 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:09:24.993608 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:09:24.993608 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:09:24.993608 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:09:24.993608 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:09:24.994756 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:09:24.994756 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:09:24.994756 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:09:24.994756 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:09:24.994756 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:09:24.994756 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:09:24.994756 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:09:24.994756 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Aug 24 09:09:24.994756 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:09:24.994756 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:09:24.994756 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:09:24.994756 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:09:24.994756 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:09:24.994756 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:09:24.994756 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:09:24.994756 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:09:24.994756 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:09:24.995928 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:09:24.995928 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:09:24.995928 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Aug 24 09:09:24.997075 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 535/1068] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 24 09:09:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:09:25.002678 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22a1ec9-f32a-41b7-96f6-283fc57c8e76 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:25.053593 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:25.104728 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22a1ec9-f32a-41b7-96f6-283fc57c8e76 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:25.138725 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814f8036-7ca5-42a6-abcd-099e3b1288c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:25.139346 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814f8036-7ca5-42a6-abcd-099e3b1288c7 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:25.139652 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814f8036-7ca5-42a6-abcd-099e3b1288c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:25.139930 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814f8036-7ca5-42a6-abcd-099e3b1288c7 None admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:25.140573 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814f8036-7ca5-42a6-abcd-099e3b1288c7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:09:25.140881 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814f8036-7ca5-42a6-abcd-099e3b1288c7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:25.142241 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814f8036-7ca5-42a6-abcd-099e3b1288c7 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:25.155801 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 534/1069] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Aug 24 09:09:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:09:25.161192 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c92dbb-9e29-4300-857b-9f43e857a293 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:25.164752 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:25.267375 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c92dbb-9e29-4300-857b-9f43e857a293 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:25.306384 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c57ec80-20e7-43d2-8c11-64bdec8d22d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:25.307061 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c57ec80-20e7-43d2-8c11-64bdec8d22d3 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:25.307407 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c57ec80-20e7-43d2-8c11-64bdec8d22d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:25.307781 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c57ec80-20e7-43d2-8c11-64bdec8d22d3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:25.314535 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c57ec80-20e7-43d2-8c11-64bdec8d22d3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:25.316080 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c57ec80-20e7-43d2-8c11-64bdec8d22d3 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:25.327807 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-6c57ec80-20e7-43d2-8c11-64bdec8d22d3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:09:25.327807 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:09:25.327807 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:09:25.327807 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:09:25.327807 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:09:25.327807 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:09:25.327807 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:09:25.327807 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:09:25.327807 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:09:25.327807 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:09:25.327807 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:09:25.327807 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:09:25.327807 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 09:09:25.327807 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 09:09:25.327807 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 09:09:25.329073 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 09:09:25.329073 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:09:25.329073 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:09:25.329073 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:09:25.329073 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:09:25.329073 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:09:25.329073 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:09:25.329073 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:09:25.329073 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Aug 24 09:09:25.329073 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:09:25.329073 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:09:25.329073 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:09:25.329073 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:09:25.329073 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:09:25.329073 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:09:25.329073 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:09:25.329073 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:09:25.330018 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 09:09:25.330018 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 09:09:25.330018 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 09:09:25.330018 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 09:09:25.330018 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 09:09:25.330018 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 09:09:25.330018 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:09:25.330018 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Aug 24 09:09:25.333488 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 536/1070] 10.222.0.16 () {56 vars in 1111 bytes} [Wed Aug 24 09:09:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:09:25.334752 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814f8036-7ca5-42a6-abcd-099e3b1288c7 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:25.435455 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814f8036-7ca5-42a6-abcd-099e3b1288c7 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:25.444939 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:25.469560 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cce66a-b4a4-4760-b8dc-57742bf752fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:25.470166 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cce66a-b4a4-4760-b8dc-57742bf752fb None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:25.470441 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cce66a-b4a4-4760-b8dc-57742bf752fb None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:25.470714 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cce66a-b4a4-4760-b8dc-57742bf752fb None admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:25.471406 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5cce66a-b4a4-4760-b8dc-57742bf752fb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:09:25.471744 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5cce66a-b4a4-4760-b8dc-57742bf752fb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:25.473169 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5cce66a-b4a4-4760-b8dc-57742bf752fb None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:25.485751 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 535/1071] 10.222.0.16 () {56 vars in 1148 bytes} [Wed Aug 24 09:09:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:09:25.493845 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c57ec80-20e7-43d2-8c11-64bdec8d22d3 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:25.556204 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:25.596711 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c57ec80-20e7-43d2-8c11-64bdec8d22d3 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:25.631493 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ccd207-0e15-4d26-879a-11947625e149 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:25.632192 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ccd207-0e15-4d26-879a-11947625e149 None admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:25.632475 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ccd207-0e15-4d26-879a-11947625e149 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:25.632751 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ccd207-0e15-4d26-879a-11947625e149 None admin] PATH_INFO: `/v3/projects/c467e88e2fb0491bacefcf6d2ce326d9/users/1f055f7dc51d43bda9b315ca67f11d58/roles/716e298774d64bc89a265236b7b9db7f` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:25.634920 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ccd207-0e15-4d26-879a-11947625e149 None admin] RBAC: Authorizing `identity:create_grant(project_id=c467e88e2fb0491bacefcf6d2ce326d9, user_id=1f055f7dc51d43bda9b315ca67f11d58, role_id=716e298774d64bc89a265236b7b9db7f)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:25.636390 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ccd207-0e15-4d26-879a-11947625e149 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:25.709298 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 537/1072] 10.222.0.16 () {58 vars in 1439 bytes} [Wed Aug 24 09:09:25 2022] PUT /identity/v3/projects/c467e88e2fb0491bacefcf6d2ce326d9/users/1f055f7dc51d43bda9b315ca67f11d58/roles/716e298774d64bc89a265236b7b9db7f => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 09:09:26.024208 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a78eba-26bd-459d-bd9e-4320dfe8c0cc None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:26.024405 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a78eba-26bd-459d-bd9e-4320dfe8c0cc None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:26.024676 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a78eba-26bd-459d-bd9e-4320dfe8c0cc None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:26.026175 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 536/1073] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:09:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:09:26.031322 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde17acb-efc8-4144-9605-f4a5fdb6fbd0 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:26.031636 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde17acb-efc8-4144-9605-f4a5fdb6fbd0 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:26.031917 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde17acb-efc8-4144-9605-f4a5fdb6fbd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:26.050640 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-dde17acb-efc8-4144-9605-f4a5fdb6fbd0 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:09:26.051880 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-dde17acb-efc8-4144-9605-f4a5fdb6fbd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:09:26.087195 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaacd1ab-9bb5-457e-8516-4313763761e1 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:26.087484 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaacd1ab-9bb5-457e-8516-4313763761e1 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:26.087780 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaacd1ab-9bb5-457e-8516-4313763761e1 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:26.089110 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 537/1074] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:09:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 09:09:26.093956 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9070ba87-19f5-47e2-89b0-d1877219170c None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:26.094236 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9070ba87-19f5-47e2-89b0-d1877219170c None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:26.094509 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9070ba87-19f5-47e2-89b0-d1877219170c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:26.112728 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-9070ba87-19f5-47e2-89b0-d1877219170c None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:09:26.113901 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-9070ba87-19f5-47e2-89b0-d1877219170c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:09:26.122498 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 538/1075] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:09:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 09:09:26.181275 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 538/1076] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:09:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 09:09:26.393491 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde17acb-efc8-4144-9605-f4a5fdb6fbd0 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:26.447251 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:26.447859 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9070ba87-19f5-47e2-89b0-d1877219170c None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:26.499390 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde17acb-efc8-4144-9605-f4a5fdb6fbd0 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:26.537022 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8492908-1d70-43ad-8d23-98527a0a384b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:26.537659 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8492908-1d70-43ad-8d23-98527a0a384b None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:26.537942 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8492908-1d70-43ad-8d23-98527a0a384b None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:26.538214 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8492908-1d70-43ad-8d23-98527a0a384b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:26.545018 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:26.549449 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9070ba87-19f5-47e2-89b0-d1877219170c None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:26.549850 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8492908-1d70-43ad-8d23-98527a0a384b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:09:26.555051 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-b8492908-1d70-43ad-8d23-98527a0a384b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:09:26.555051 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:09:26.555051 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:09:26.555051 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:09:26.555051 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:09:26.555051 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:09:26.555051 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:09:26.555051 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:09:26.555051 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:09:26.555051 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:09:26.555051 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:09:26.555051 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:09:26.555051 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:09:26.555051 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:09:26.555051 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:09:26.555051 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:09:26.556714 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:09:26.556714 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:09:26.556714 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:09:26.556714 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:09:26.556714 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:09:26.556714 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:09:26.556714 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:09:26.556714 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:09:26.556714 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:09:26.556714 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:09:26.556714 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:09:26.556714 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Aug 24 09:09:26.556714 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:09:26.556714 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:09:26.556714 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:09:26.556714 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:09:26.557577 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:09:26.557577 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:09:26.557577 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:09:26.557577 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:09:26.557577 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:09:26.557577 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:09:26.557577 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:09:26.557577 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:09:26.557577 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:09:26.557577 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:09:26.557577 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:09:26.557577 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Aug 24 09:09:26.559911 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 539/1077] 10.222.0.16 () {56 vars in 1096 bytes} [Wed Aug 24 09:09:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:09:26.561942 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8492908-1d70-43ad-8d23-98527a0a384b None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:26.587018 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bcb35d-077b-405c-8ecb-079fdfb12b59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:26.587663 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bcb35d-077b-405c-8ecb-079fdfb12b59 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:26.587952 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bcb35d-077b-405c-8ecb-079fdfb12b59 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:26.588218 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bcb35d-077b-405c-8ecb-079fdfb12b59 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:26.660110 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bcb35d-077b-405c-8ecb-079fdfb12b59 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:26.661551 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bcb35d-077b-405c-8ecb-079fdfb12b59 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:26.673064 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8492908-1d70-43ad-8d23-98527a0a384b None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:26.688013 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:26.711505 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1c94f5-06d7-43c7-8587-30df47d9c9bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:26.712142 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1c94f5-06d7-43c7-8587-30df47d9c9bd None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:26.712423 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1c94f5-06d7-43c7-8587-30df47d9c9bd None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:26.712697 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1c94f5-06d7-43c7-8587-30df47d9c9bd None admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:26.713364 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1c94f5-06d7-43c7-8587-30df47d9c9bd None admin] RBAC: Adding query filter params (name=admin) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:09:26.713674 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1c94f5-06d7-43c7-8587-30df47d9c9bd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:26.715043 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1c94f5-06d7-43c7-8587-30df47d9c9bd None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:26.727708 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 540/1078] 10.222.0.16 () {56 vars in 1099 bytes} [Wed Aug 24 09:09:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:09:26.730335 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1c94f5-06d7-43c7-8587-30df47d9c9bd None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:26.765146 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:26.769250 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 539/1079] 10.222.0.16 () {58 vars in 1282 bytes} [Wed Aug 24 09:09:26 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 24 09:09:26.775379 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bcb35d-077b-405c-8ecb-079fdfb12b59 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:26.841086 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1c94f5-06d7-43c7-8587-30df47d9c9bd None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:26.877416 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e05e16-4977-4764-99cc-e9dedaf28d84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:26.878045 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e05e16-4977-4764-99cc-e9dedaf28d84 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:26.878242 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bcb35d-077b-405c-8ecb-079fdfb12b59 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:26.878350 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e05e16-4977-4764-99cc-e9dedaf28d84 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:26.878578 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e05e16-4977-4764-99cc-e9dedaf28d84 None admin] PATH_INFO: `/v3/users/b3dab487cd8a43ac9a7760c8df3a2b7c` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:26.879733 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e05e16-4977-4764-99cc-e9dedaf28d84 None admin] RBAC: Authorizing `identity:get_user(user_id=b3dab487cd8a43ac9a7760c8df3a2b7c)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:26.881159 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e05e16-4977-4764-99cc-e9dedaf28d84 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:26.883551 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 541/1080] 10.222.0.16 () {56 vars in 1177 bytes} [Wed Aug 24 09:09:26 2022] GET /identity/v3/users/b3dab487cd8a43ac9a7760c8df3a2b7c => generated 285 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:09:26.888908 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e05e16-4977-4764-99cc-e9dedaf28d84 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:26.918834 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4da1bf-0e35-400e-ade0-8a44f87afe6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:26.919455 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4da1bf-0e35-400e-ade0-8a44f87afe6f None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:26.919788 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4da1bf-0e35-400e-ade0-8a44f87afe6f None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:26.920027 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4da1bf-0e35-400e-ade0-8a44f87afe6f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:26.930707 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4da1bf-0e35-400e-ade0-8a44f87afe6f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:26.932208 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4da1bf-0e35-400e-ade0-8a44f87afe6f None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:26.933432 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:26.947375 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-2e4da1bf-0e35-400e-ade0-8a44f87afe6f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:09:26.947375 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:09:26.947375 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:09:26.947375 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:09:26.947375 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:09:26.947375 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:09:26.947375 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:09:26.947375 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:09:26.947375 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:09:26.947375 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:09:26.947375 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:09:26.947375 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:09:26.947375 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:09:26.947375 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:09:26.947375 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:09:26.947375 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:09:26.948387 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:09:26.948387 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:09:26.948387 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:09:26.948387 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:09:26.948387 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:09:26.948387 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:09:26.948387 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:09:26.948387 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 24 09:09:26.948387 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:09:26.948387 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:09:26.948387 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:09:26.948387 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:09:26.948387 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:09:26.948387 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:09:26.948387 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:09:26.948387 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:09:26.948387 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:09:26.949317 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:09:26.949317 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:09:26.949317 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 24 09:09:26.949317 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 540/1081] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 24 09:09:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 09:09:26.954859 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4da1bf-0e35-400e-ade0-8a44f87afe6f None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:26.995007 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e05e16-4977-4764-99cc-e9dedaf28d84 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:27.006799 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:27.030965 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4c04fa-71e9-49ec-8330-d673195c1e08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:27.031613 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4c04fa-71e9-49ec-8330-d673195c1e08 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:27.031908 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4c04fa-71e9-49ec-8330-d673195c1e08 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:27.032184 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4c04fa-71e9-49ec-8330-d673195c1e08 None admin] PATH_INFO: `/v3/projects/e8ae36cb945c488682ab38083533ff07` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:27.033427 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4c04fa-71e9-49ec-8330-d673195c1e08 None admin] RBAC: Authorizing `identity:get_project(project_id=e8ae36cb945c488682ab38083533ff07)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:27.034869 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4c04fa-71e9-49ec-8330-d673195c1e08 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:27.041795 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 542/1082] 10.222.0.16 () {56 vars in 1186 bytes} [Wed Aug 24 09:09:26 2022] GET /identity/v3/projects/e8ae36cb945c488682ab38083533ff07 => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:09:27.043210 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4c04fa-71e9-49ec-8330-d673195c1e08 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:27.062031 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4da1bf-0e35-400e-ade0-8a44f87afe6f None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:27.100752 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ec10b9-66d3-41cd-8636-e3177b1ccbe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:27.101289 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ec10b9-66d3-41cd-8636-e3177b1ccbe1 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:27.102152 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ec10b9-66d3-41cd-8636-e3177b1ccbe1 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:27.102527 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ec10b9-66d3-41cd-8636-e3177b1ccbe1 None admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:27.103163 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ec10b9-66d3-41cd-8636-e3177b1ccbe1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:09:27.103465 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ec10b9-66d3-41cd-8636-e3177b1ccbe1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:27.105099 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ec10b9-66d3-41cd-8636-e3177b1ccbe1 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:27.127621 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 541/1083] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Aug 24 09:09:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:09:27.157507 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4c04fa-71e9-49ec-8330-d673195c1e08 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:27.170481 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:27.196964 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceeca81b-9089-44de-834a-41e806de1600 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:27.197794 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceeca81b-9089-44de-834a-41e806de1600 None admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:27.198081 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceeca81b-9089-44de-834a-41e806de1600 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:27.198359 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceeca81b-9089-44de-834a-41e806de1600 None admin] PATH_INFO: `/v3/projects/e8ae36cb945c488682ab38083533ff07/users/b3dab487cd8a43ac9a7760c8df3a2b7c/roles/15f634d4a2d247ddab90c458e5e1c141` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:27.200423 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceeca81b-9089-44de-834a-41e806de1600 None admin] RBAC: Authorizing `identity:create_grant(project_id=e8ae36cb945c488682ab38083533ff07, user_id=b3dab487cd8a43ac9a7760c8df3a2b7c, role_id=15f634d4a2d247ddab90c458e5e1c141)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:27.201887 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceeca81b-9089-44de-834a-41e806de1600 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:27.303404 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 543/1084] 10.222.0.16 () {58 vars in 1439 bytes} [Wed Aug 24 09:09:27 2022] PUT /identity/v3/projects/e8ae36cb945c488682ab38083533ff07/users/b3dab487cd8a43ac9a7760c8df3a2b7c/roles/15f634d4a2d247ddab90c458e5e1c141 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 09:09:27.623016 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7809b69-91ba-4b3f-872b-132f54fd4846 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:27.623543 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7809b69-91ba-4b3f-872b-132f54fd4846 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:27.623543 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7809b69-91ba-4b3f-872b-132f54fd4846 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:27.624927 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 542/1085] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:09:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:09:27.630130 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541c9e43-8923-4825-9ee0-7fd2a178078c None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:27.630567 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541c9e43-8923-4825-9ee0-7fd2a178078c None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:27.630688 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541c9e43-8923-4825-9ee0-7fd2a178078c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:27.649962 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-541c9e43-8923-4825-9ee0-7fd2a178078c None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:09:27.651186 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-541c9e43-8923-4825-9ee0-7fd2a178078c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:09:27.717736 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 544/1086] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:09:27 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 09:09:27.991516 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7809b69-91ba-4b3f-872b-132f54fd4846 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:28.092273 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7809b69-91ba-4b3f-872b-132f54fd4846 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:28.103038 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:28.125223 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515c6d80-68e3-4faa-beca-6d62c8bb84bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:28.125842 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515c6d80-68e3-4faa-beca-6d62c8bb84bf None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:28.126117 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515c6d80-68e3-4faa-beca-6d62c8bb84bf None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:28.126393 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515c6d80-68e3-4faa-beca-6d62c8bb84bf None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:28.136268 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515c6d80-68e3-4faa-beca-6d62c8bb84bf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:09:28.140917 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-515c6d80-68e3-4faa-beca-6d62c8bb84bf None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 09:09:28.140917 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:09:28.140917 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:09:28.140917 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:09:28.140917 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:09:28.140917 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:09:28.140917 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:09:28.140917 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:09:28.140917 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:09:28.140917 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:09:28.140917 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:09:28.140917 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:09:28.140917 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:09:28.140917 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:09:28.140917 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:09:28.140917 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:09:28.141899 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:09:28.141899 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:09:28.141899 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:09:28.141899 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:09:28.141899 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:09:28.141899 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:09:28.141899 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:09:28.141899 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:09:28.141899 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:09:28.141899 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:09:28.141899 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:09:28.141899 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 24 09:09:28.141899 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:09:28.141899 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:09:28.141899 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:09:28.141899 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:09:28.142749 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:09:28.142749 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:09:28.142749 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:09:28.142749 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:09:28.142749 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:09:28.142749 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:09:28.142749 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:09:28.142749 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:09:28.142749 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:09:28.142749 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:09:28.142749 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 09:09:28.142749 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 24 09:09:28.142749 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 543/1087] 10.222.0.16 () {56 vars in 1120 bytes} [Wed Aug 24 09:09:27 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:09:28.147967 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541c9e43-8923-4825-9ee0-7fd2a178078c None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:28.273971 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:28.276742 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541c9e43-8923-4825-9ee0-7fd2a178078c None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:28.310988 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd37dc4-095f-43b9-8d5b-ca1469827dfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:28.311627 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd37dc4-095f-43b9-8d5b-ca1469827dfe None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:28.311910 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd37dc4-095f-43b9-8d5b-ca1469827dfe None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:28.312180 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd37dc4-095f-43b9-8d5b-ca1469827dfe None admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:28.312841 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd37dc4-095f-43b9-8d5b-ca1469827dfe None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:09:28.313142 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd37dc4-095f-43b9-8d5b-ca1469827dfe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:28.314498 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd37dc4-095f-43b9-8d5b-ca1469827dfe None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:28.322254 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 545/1088] 10.222.0.16 () {56 vars in 1115 bytes} [Wed Aug 24 09:09:28 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:09:28.328053 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515c6d80-68e3-4faa-beca-6d62c8bb84bf None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:28.430522 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515c6d80-68e3-4faa-beca-6d62c8bb84bf None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:28.463016 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a0627e-0a62-4a98-8e95-b47344636574 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:28.463726 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a0627e-0a62-4a98-8e95-b47344636574 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:28.464009 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a0627e-0a62-4a98-8e95-b47344636574 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:28.464278 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a0627e-0a62-4a98-8e95-b47344636574 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:28.474451 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a0627e-0a62-4a98-8e95-b47344636574 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:28.475917 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a0627e-0a62-4a98-8e95-b47344636574 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:28.490015 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-07a0627e-0a62-4a98-8e95-b47344636574 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:09:28.490015 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:09:28.490015 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:09:28.490015 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:09:28.490015 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:09:28.490015 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:09:28.490015 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:09:28.490015 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:09:28.490015 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:09:28.490015 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:09:28.490015 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:09:28.490015 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:09:28.490015 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:09:28.490015 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:09:28.490015 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:09:28.490015 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:09:28.490944 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:09:28.490944 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:09:28.490944 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:09:28.490944 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:09:28.490944 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:09:28.490944 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:09:28.490944 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:09:28.490944 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 24 09:09:28.490944 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:09:28.490944 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:09:28.490944 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:09:28.490944 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:09:28.490944 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:09:28.490944 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:09:28.490944 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:09:28.490944 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:09:28.490944 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:09:28.492776 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:09:28.492776 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:09:28.492776 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 24 09:09:28.492776 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 544/1089] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 24 09:09:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:09:28.497593 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd37dc4-095f-43b9-8d5b-ca1469827dfe None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:28.595126 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd37dc4-095f-43b9-8d5b-ca1469827dfe None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:28.628693 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e93ece-9d28-41cf-be03-49e2ee6fa109 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:28.629293 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e93ece-9d28-41cf-be03-49e2ee6fa109 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:28.629573 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e93ece-9d28-41cf-be03-49e2ee6fa109 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:28.629842 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e93ece-9d28-41cf-be03-49e2ee6fa109 None admin] PATH_INFO: `/v3/domains` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:28.630476 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e93ece-9d28-41cf-be03-49e2ee6fa109 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:09:28.631020 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e93ece-9d28-41cf-be03-49e2ee6fa109 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:28.632151 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e93ece-9d28-41cf-be03-49e2ee6fa109 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:28.643654 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 546/1090] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Aug 24 09:09:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:09:28.649403 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a0627e-0a62-4a98-8e95-b47344636574 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:28.674515 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:28.749306 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a0627e-0a62-4a98-8e95-b47344636574 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:28.782403 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e5d98e-b04f-4ff5-a459-e05dd781d1bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:28.783030 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e5d98e-b04f-4ff5-a459-e05dd781d1bb None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:28.783307 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e5d98e-b04f-4ff5-a459-e05dd781d1bb None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:28.783610 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e5d98e-b04f-4ff5-a459-e05dd781d1bb None admin] PATH_INFO: `/v3/users/nova` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:28.790991 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e5d98e-b04f-4ff5-a459-e05dd781d1bb None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:28.792467 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e5d98e-b04f-4ff5-a459-e05dd781d1bb None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:28.804383 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-a8e5d98e-b04f-4ff5-a459-e05dd781d1bb None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 09:09:28.804383 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:09:28.804383 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:09:28.804383 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:09:28.804383 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:09:28.804383 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:09:28.804383 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:09:28.804383 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:09:28.804383 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:09:28.804383 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:09:28.804383 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:09:28.804383 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:09:28.804383 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 09:09:28.804383 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 09:09:28.804383 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 09:09:28.804383 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 09:09:28.805328 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:09:28.805328 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:09:28.805328 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 09:09:28.805328 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:09:28.805328 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 09:09:28.805328 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:09:28.805328 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:09:28.805328 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:09:28.805328 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:09:28.805328 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:09:28.805328 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:09:28.805328 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 24 09:09:28.805328 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:09:28.805328 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:09:28.805328 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:09:28.805328 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:09:28.805328 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:09:28.805328 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:09:28.806676 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:09:28.806676 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:09:28.806676 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 09:09:28.806676 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 09:09:28.806676 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 09:09:28.806676 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 09:09:28.806676 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 09:09:28.806676 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 09:09:28.806676 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 09:09:28.806676 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 24 09:09:28.806676 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 545/1091] 10.222.0.16 () {56 vars in 1093 bytes} [Wed Aug 24 09:09:28 2022] GET /identity/v3/users/nova => generated 82 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:09:28.811294 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e93ece-9d28-41cf-be03-49e2ee6fa109 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:28.835130 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:28.908639 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e93ece-9d28-41cf-be03-49e2ee6fa109 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:28.945113 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0f8fa0-82da-455d-abd5-6e229bdb8f09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:28.945730 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0f8fa0-82da-455d-abd5-6e229bdb8f09 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:28.946010 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0f8fa0-82da-455d-abd5-6e229bdb8f09 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:28.946282 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0f8fa0-82da-455d-abd5-6e229bdb8f09 None admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:28.947006 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0f8fa0-82da-455d-abd5-6e229bdb8f09 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:09:28.947431 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0f8fa0-82da-455d-abd5-6e229bdb8f09 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:28.948895 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0f8fa0-82da-455d-abd5-6e229bdb8f09 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:28.969543 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 547/1092] 10.222.0.16 () {56 vars in 1133 bytes} [Wed Aug 24 09:09:28 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:09:28.975349 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e5d98e-b04f-4ff5-a459-e05dd781d1bb None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:29.055738 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:29.073106 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e5d98e-b04f-4ff5-a459-e05dd781d1bb None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:29.108039 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f4cb8a-c9c6-4c20-965a-5e6b3c37194e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:29.108673 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f4cb8a-c9c6-4c20-965a-5e6b3c37194e None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:29.108953 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f4cb8a-c9c6-4c20-965a-5e6b3c37194e None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:29.109226 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f4cb8a-c9c6-4c20-965a-5e6b3c37194e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:29.175074 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f4cb8a-c9c6-4c20-965a-5e6b3c37194e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:29.176482 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f4cb8a-c9c6-4c20-965a-5e6b3c37194e None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:29.178336 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ce6378-57e9-44b7-b37a-4986dd9aecd9 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:29.178688 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ce6378-57e9-44b7-b37a-4986dd9aecd9 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:29.179039 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ce6378-57e9-44b7-b37a-4986dd9aecd9 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:29.180479 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 548/1093] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:09:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 09:09:29.185598 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2e4949-24f4-4e34-ae5d-c72d1459bb8a None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:29.185881 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2e4949-24f4-4e34-ae5d-c72d1459bb8a None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:29.186153 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2e4949-24f4-4e34-ae5d-c72d1459bb8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:29.204701 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-6d2e4949-24f4-4e34-ae5d-c72d1459bb8a None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:09:29.205903 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-6d2e4949-24f4-4e34-ae5d-c72d1459bb8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:09:29.256485 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:29.272350 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 549/1094] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:09:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 09:09:29.280577 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 546/1095] 10.222.0.16 () {58 vars in 1282 bytes} [Wed Aug 24 09:09:28 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 09:09:29.281860 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2e4949-24f4-4e34-ae5d-c72d1459bb8a None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:29.294362 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:29.381731 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2e4949-24f4-4e34-ae5d-c72d1459bb8a None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:29.415936 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae47c565-042f-4151-97a0-25958b012414 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:29.416589 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae47c565-042f-4151-97a0-25958b012414 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:29.416872 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae47c565-042f-4151-97a0-25958b012414 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:29.417144 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae47c565-042f-4151-97a0-25958b012414 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:29.426894 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae47c565-042f-4151-97a0-25958b012414 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:29.428310 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae47c565-042f-4151-97a0-25958b012414 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:29.443005 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-ae47c565-042f-4151-97a0-25958b012414 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:09:29.443005 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:09:29.443005 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:09:29.443005 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:09:29.443005 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:09:29.443005 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:09:29.443005 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:09:29.443005 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:09:29.443005 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:09:29.443005 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:09:29.443005 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:09:29.443005 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:09:29.443005 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:09:29.443005 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:09:29.443005 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:09:29.443005 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:09:29.444149 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:09:29.444149 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:09:29.444149 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:09:29.444149 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:09:29.444149 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:09:29.444149 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:09:29.444149 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:09:29.444149 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Aug 24 09:09:29.444149 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:09:29.444149 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:09:29.444149 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:09:29.444149 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:09:29.444149 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:09:29.444149 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:09:29.444149 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:09:29.444149 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:09:29.444149 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:09:29.445246 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:09:29.445246 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:09:29.445246 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Aug 24 09:09:29.447730 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 550/1096] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 24 09:09:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:09:29.449951 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f4cb8a-c9c6-4c20-965a-5e6b3c37194e None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:29.538731 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae47c565-042f-4151-97a0-25958b012414 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:29.555836 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f4cb8a-c9c6-4c20-965a-5e6b3c37194e None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:29.589286 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068de395-4267-4c21-aadb-53836cd04ba8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:29.589896 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068de395-4267-4c21-aadb-53836cd04ba8 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:29.590179 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068de395-4267-4c21-aadb-53836cd04ba8 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:29.590457 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068de395-4267-4c21-aadb-53836cd04ba8 None admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:29.591130 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068de395-4267-4c21-aadb-53836cd04ba8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:09:29.591440 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068de395-4267-4c21-aadb-53836cd04ba8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:29.592850 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068de395-4267-4c21-aadb-53836cd04ba8 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:29.600222 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:29.604847 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 547/1097] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Aug 24 09:09:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:09:29.610593 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068de395-4267-4c21-aadb-53836cd04ba8 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:29.640844 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae47c565-042f-4151-97a0-25958b012414 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:29.652741 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:29.674167 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412d40ad-c6b1-4b45-94c6-72c8fac4e76d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:29.674790 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412d40ad-c6b1-4b45-94c6-72c8fac4e76d None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:29.675067 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412d40ad-c6b1-4b45-94c6-72c8fac4e76d None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:29.675333 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412d40ad-c6b1-4b45-94c6-72c8fac4e76d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:29.686567 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412d40ad-c6b1-4b45-94c6-72c8fac4e76d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:09:29.693162 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-412d40ad-c6b1-4b45-94c6-72c8fac4e76d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:09:29.693162 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:09:29.693162 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:09:29.693162 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:09:29.693162 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:09:29.693162 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:09:29.693162 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:09:29.693162 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:09:29.693162 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:09:29.693162 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:09:29.693162 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:09:29.693162 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:09:29.693162 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:09:29.693162 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:09:29.693162 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:09:29.693162 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:09:29.694295 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:09:29.694295 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:09:29.694295 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:09:29.694295 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:09:29.694295 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:09:29.694295 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:09:29.694295 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:09:29.694295 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:09:29.694295 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:09:29.694295 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:09:29.694295 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:09:29.694295 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Aug 24 09:09:29.694295 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:09:29.694295 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:09:29.694295 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:09:29.694295 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:09:29.695180 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:09:29.695180 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:09:29.695180 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:09:29.695180 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:09:29.695180 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:09:29.695180 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:09:29.695180 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:09:29.695180 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:09:29.695180 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:09:29.695180 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:09:29.695180 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:09:29.695180 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Aug 24 09:09:29.695180 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 551/1098] 10.222.0.16 () {56 vars in 1096 bytes} [Wed Aug 24 09:09:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 09:09:29.700340 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412d40ad-c6b1-4b45-94c6-72c8fac4e76d None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:29.711769 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068de395-4267-4c21-aadb-53836cd04ba8 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:29.745074 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e079660-7e0f-4bab-8a2b-ffa8195fc34b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:29.745707 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e079660-7e0f-4bab-8a2b-ffa8195fc34b None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:29.745993 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e079660-7e0f-4bab-8a2b-ffa8195fc34b None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:29.746269 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e079660-7e0f-4bab-8a2b-ffa8195fc34b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:29.752816 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e079660-7e0f-4bab-8a2b-ffa8195fc34b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:29.754268 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e079660-7e0f-4bab-8a2b-ffa8195fc34b None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:29.765617 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-3e079660-7e0f-4bab-8a2b-ffa8195fc34b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:09:29.765617 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:09:29.765617 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:09:29.765617 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:09:29.765617 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:09:29.765617 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:09:29.765617 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:09:29.765617 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:09:29.765617 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:09:29.765617 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:09:29.765617 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:09:29.765617 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:09:29.765617 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 09:09:29.765617 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 09:09:29.765617 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 09:09:29.771678 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 09:09:29.771678 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:09:29.771678 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:09:29.771678 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:09:29.771678 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:09:29.771678 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:09:29.771678 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:09:29.771678 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:09:29.771678 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 24 09:09:29.771678 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:09:29.771678 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:09:29.771678 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:09:29.771678 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:09:29.771678 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:09:29.771678 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:09:29.771678 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:09:29.771678 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:09:29.772657 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 09:09:29.772657 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 09:09:29.772657 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 09:09:29.772657 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 09:09:29.772657 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 09:09:29.772657 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 09:09:29.772657 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:09:29.772657 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 24 09:09:29.772657 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 548/1099] 10.222.0.16 () {56 vars in 1111 bytes} [Wed Aug 24 09:09:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 09:09:29.772657 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e079660-7e0f-4bab-8a2b-ffa8195fc34b None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:29.798503 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412d40ad-c6b1-4b45-94c6-72c8fac4e76d None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:29.820857 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:29.832483 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbebbf40-44c4-4cd7-bba1-b460d983e822 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:29.833088 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbebbf40-44c4-4cd7-bba1-b460d983e822 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:29.833373 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbebbf40-44c4-4cd7-bba1-b460d983e822 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:29.833648 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbebbf40-44c4-4cd7-bba1-b460d983e822 None admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:29.834314 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbebbf40-44c4-4cd7-bba1-b460d983e822 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:09:29.834619 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbebbf40-44c4-4cd7-bba1-b460d983e822 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:29.836668 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbebbf40-44c4-4cd7-bba1-b460d983e822 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:29.844525 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 552/1100] 10.222.0.16 () {56 vars in 1099 bytes} [Wed Aug 24 09:09:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:09:29.850455 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbebbf40-44c4-4cd7-bba1-b460d983e822 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:29.872997 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e079660-7e0f-4bab-8a2b-ffa8195fc34b None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:29.874492 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:29.907691 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da181d51-f64c-4841-bbb6-cde7ae47cfdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:29.908294 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da181d51-f64c-4841-bbb6-cde7ae47cfdb None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:29.908577 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da181d51-f64c-4841-bbb6-cde7ae47cfdb None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:29.908851 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da181d51-f64c-4841-bbb6-cde7ae47cfdb None admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:29.909543 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da181d51-f64c-4841-bbb6-cde7ae47cfdb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:09:29.909845 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da181d51-f64c-4841-bbb6-cde7ae47cfdb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:29.911259 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da181d51-f64c-4841-bbb6-cde7ae47cfdb None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:29.923015 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 549/1101] 10.222.0.16 () {56 vars in 1148 bytes} [Wed Aug 24 09:09:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:09:29.929130 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da181d51-f64c-4841-bbb6-cde7ae47cfdb None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:29.949545 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbebbf40-44c4-4cd7-bba1-b460d983e822 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:29.983103 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13307f2e-99bd-43e2-bdeb-416789bd243c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:29.983774 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13307f2e-99bd-43e2-bdeb-416789bd243c None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:29.984052 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13307f2e-99bd-43e2-bdeb-416789bd243c None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:29.984323 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13307f2e-99bd-43e2-bdeb-416789bd243c None admin] PATH_INFO: `/v3/users/b3dab487cd8a43ac9a7760c8df3a2b7c` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:29.985447 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13307f2e-99bd-43e2-bdeb-416789bd243c None admin] RBAC: Authorizing `identity:get_user(user_id=b3dab487cd8a43ac9a7760c8df3a2b7c)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:29.986867 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13307f2e-99bd-43e2-bdeb-416789bd243c None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:29.988463 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 553/1102] 10.222.0.16 () {56 vars in 1177 bytes} [Wed Aug 24 09:09:29 2022] GET /identity/v3/users/b3dab487cd8a43ac9a7760c8df3a2b7c => generated 285 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:09:29.994800 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13307f2e-99bd-43e2-bdeb-416789bd243c None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:30.031290 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da181d51-f64c-4841-bbb6-cde7ae47cfdb None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:30.046312 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:30.064966 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f049ba6-25b9-4736-8d30-a132ee18fd98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:30.065564 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f049ba6-25b9-4736-8d30-a132ee18fd98 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:30.065841 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f049ba6-25b9-4736-8d30-a132ee18fd98 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:30.066109 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f049ba6-25b9-4736-8d30-a132ee18fd98 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:30.066831 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f049ba6-25b9-4736-8d30-a132ee18fd98 None admin] RBAC: Adding query filter params (role.id=716e298774d64bc89a265236b7b9db7f, scope.project.id=c467e88e2fb0491bacefcf6d2ce326d9, user.id=1f055f7dc51d43bda9b315ca67f11d58) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:09:30.067138 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f049ba6-25b9-4736-8d30-a132ee18fd98 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:30.068597 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f049ba6-25b9-4736-8d30-a132ee18fd98 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:30.087610 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 550/1103] 10.222.0.16 () {56 vars in 1374 bytes} [Wed Aug 24 09:09:29 2022] GET /identity/v3/role_assignments?user.id=1f055f7dc51d43bda9b315ca67f11d58&scope.project.id=c467e88e2fb0491bacefcf6d2ce326d9&role.id=716e298774d64bc89a265236b7b9db7f => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:09:30.093045 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13307f2e-99bd-43e2-bdeb-416789bd243c None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:30.130034 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5012705-c8f4-4bc4-bb65-f3bb44e4b5a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:30.130656 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5012705-c8f4-4bc4-bb65-f3bb44e4b5a0 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:30.130933 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5012705-c8f4-4bc4-bb65-f3bb44e4b5a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:30.131201 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5012705-c8f4-4bc4-bb65-f3bb44e4b5a0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:30.202389 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5012705-c8f4-4bc4-bb65-f3bb44e4b5a0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:30.204056 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5012705-c8f4-4bc4-bb65-f3bb44e4b5a0 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:30.256317 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:30.307988 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 554/1104] 10.222.0.16 () {58 vars in 1282 bytes} [Wed Aug 24 09:09:29 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 24 09:09:30.313881 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f049ba6-25b9-4736-8d30-a132ee18fd98 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:30.347828 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:30.424078 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f049ba6-25b9-4736-8d30-a132ee18fd98 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:30.457663 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f815aa9-f589-45fa-8409-c3c73daebccd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:30.458331 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f815aa9-f589-45fa-8409-c3c73daebccd None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:30.458511 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f815aa9-f589-45fa-8409-c3c73daebccd None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:30.458783 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f815aa9-f589-45fa-8409-c3c73daebccd None admin] PATH_INFO: `/v3/projects/e8ae36cb945c488682ab38083533ff07` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:30.459927 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f815aa9-f589-45fa-8409-c3c73daebccd None admin] RBAC: Authorizing `identity:get_project(project_id=e8ae36cb945c488682ab38083533ff07)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:30.461902 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f815aa9-f589-45fa-8409-c3c73daebccd None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:30.463545 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 551/1105] 10.222.0.16 () {56 vars in 1186 bytes} [Wed Aug 24 09:09:30 2022] GET /identity/v3/projects/e8ae36cb945c488682ab38083533ff07 => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:09:30.469693 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5012705-c8f4-4bc4-bb65-f3bb44e4b5a0 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:30.573618 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5012705-c8f4-4bc4-bb65-f3bb44e4b5a0 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:30.609779 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779f0bdd-09f1-49a6-b69d-fd38560cd46c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:30.610391 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779f0bdd-09f1-49a6-b69d-fd38560cd46c None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:30.610671 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779f0bdd-09f1-49a6-b69d-fd38560cd46c None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:30.610945 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779f0bdd-09f1-49a6-b69d-fd38560cd46c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:30.611705 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779f0bdd-09f1-49a6-b69d-fd38560cd46c None admin] RBAC: Adding query filter params (role.id=15f634d4a2d247ddab90c458e5e1c141, scope.project.id=e8ae36cb945c488682ab38083533ff07, user.id=b3dab487cd8a43ac9a7760c8df3a2b7c) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:09:30.612017 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779f0bdd-09f1-49a6-b69d-fd38560cd46c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:30.613439 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779f0bdd-09f1-49a6-b69d-fd38560cd46c None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:30.619641 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 555/1106] 10.222.0.16 () {56 vars in 1374 bytes} [Wed Aug 24 09:09:30 2022] GET /identity/v3/role_assignments?user.id=b3dab487cd8a43ac9a7760c8df3a2b7c&scope.project.id=e8ae36cb945c488682ab38083533ff07&role.id=15f634d4a2d247ddab90c458e5e1c141 => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:09:32.541148 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1832d2a-011e-4cf8-968e-5dd9b4063a06 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:32.541689 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1832d2a-011e-4cf8-968e-5dd9b4063a06 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:32.541689 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1832d2a-011e-4cf8-968e-5dd9b4063a06 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:32.543067 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 552/1107] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:09:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:09:32.548290 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84097a4-e60e-45df-b4ac-5b3a30952b3f None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:32.548705 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84097a4-e60e-45df-b4ac-5b3a30952b3f None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:32.548838 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84097a4-e60e-45df-b4ac-5b3a30952b3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:32.567087 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-b84097a4-e60e-45df-b4ac-5b3a30952b3f None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:09:32.568228 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-b84097a4-e60e-45df-b4ac-5b3a30952b3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:09:32.578767 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:32.633403 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 556/1108] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:09:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 09:09:32.909186 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1832d2a-011e-4cf8-968e-5dd9b4063a06 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:33.001488 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:33.008956 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1832d2a-011e-4cf8-968e-5dd9b4063a06 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:33.041888 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9668c1e-2b11-4923-af70-288ab04858ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:33.042516 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9668c1e-2b11-4923-af70-288ab04858ba None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:33.042798 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9668c1e-2b11-4923-af70-288ab04858ba None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:33.043071 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9668c1e-2b11-4923-af70-288ab04858ba None admin] PATH_INFO: `/v3/domains/default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:33.044202 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9668c1e-2b11-4923-af70-288ab04858ba None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:33.045628 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9668c1e-2b11-4923-af70-288ab04858ba None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:33.047193 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 553/1109] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 24 09:09:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:09:33.052915 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84097a4-e60e-45df-b4ac-5b3a30952b3f None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:33.147207 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84097a4-e60e-45df-b4ac-5b3a30952b3f None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:33.180553 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e65fadd-9d39-497f-9372-0ba22f539807 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:33.181091 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e65fadd-9d39-497f-9372-0ba22f539807 None admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:33.181367 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e65fadd-9d39-497f-9372-0ba22f539807 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:33.181635 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e65fadd-9d39-497f-9372-0ba22f539807 None admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:33.182260 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e65fadd-9d39-497f-9372-0ba22f539807 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:33.183688 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e65fadd-9d39-497f-9372-0ba22f539807 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:33.287997 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 557/1110] 10.222.0.16 () {60 vars in 1132 bytes} [Wed Aug 24 09:09:33 2022] POST /identity/v3/users => generated 286 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:09:35.250233 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8197de0d-6a72-41ff-9182-842de9598ae1 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:35.250798 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8197de0d-6a72-41ff-9182-842de9598ae1 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:35.250798 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8197de0d-6a72-41ff-9182-842de9598ae1 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:35.252156 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 554/1111] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:09:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:09:35.257944 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dfd7a2-fe0f-427d-8903-b8969c308e56 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:35.258406 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dfd7a2-fe0f-427d-8903-b8969c308e56 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:35.258503 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dfd7a2-fe0f-427d-8903-b8969c308e56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:35.264781 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:35.277359 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-77dfd7a2-fe0f-427d-8903-b8969c308e56 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:09:35.278544 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-77dfd7a2-fe0f-427d-8903-b8969c308e56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:09:35.345646 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 558/1112] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:09:35 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 09:09:35.609615 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8197de0d-6a72-41ff-9182-842de9598ae1 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:35.707794 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8197de0d-6a72-41ff-9182-842de9598ae1 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:35.739353 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28721b2-9298-4ba7-8c82-c0fcc430f5ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:35.739990 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28721b2-9298-4ba7-8c82-c0fcc430f5ca None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:35.740288 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28721b2-9298-4ba7-8c82-c0fcc430f5ca None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:35.740565 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28721b2-9298-4ba7-8c82-c0fcc430f5ca None admin] PATH_INFO: `/v3/roles/ed6466d17fb7403c8491bb7f1ef95fc1` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:35.742465 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28721b2-9298-4ba7-8c82-c0fcc430f5ca None admin] RBAC: Authorizing `identity:get_role(role_id=ed6466d17fb7403c8491bb7f1ef95fc1)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:35.743835 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28721b2-9298-4ba7-8c82-c0fcc430f5ca None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:35.745119 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 555/1113] 10.222.0.16 () {56 vars in 1177 bytes} [Wed Aug 24 09:09:35 2022] GET /identity/v3/roles/ed6466d17fb7403c8491bb7f1ef95fc1 => generated 223 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:09:35.750943 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dfd7a2-fe0f-427d-8903-b8969c308e56 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:35.846524 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dfd7a2-fe0f-427d-8903-b8969c308e56 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:35.874770 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:35.879415 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a9860a-7767-4ba6-8f2b-91f158d1efbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:35.880065 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a9860a-7767-4ba6-8f2b-91f158d1efbc None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:35.880348 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a9860a-7767-4ba6-8f2b-91f158d1efbc None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:35.880620 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a9860a-7767-4ba6-8f2b-91f158d1efbc None admin] PATH_INFO: `/v3/users/1bcb46d61e2a491a880e14d7c5cfde7d` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:35.893650 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a9860a-7767-4ba6-8f2b-91f158d1efbc None admin] RBAC: Authorizing `identity:get_user(user_id=1bcb46d61e2a491a880e14d7c5cfde7d)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:35.895077 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a9860a-7767-4ba6-8f2b-91f158d1efbc None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:35.897025 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 559/1114] 10.222.0.16 () {56 vars in 1177 bytes} [Wed Aug 24 09:09:35 2022] GET /identity/v3/users/1bcb46d61e2a491a880e14d7c5cfde7d => generated 286 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:09:35.902462 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28721b2-9298-4ba7-8c82-c0fcc430f5ca None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:35.932520 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:36.003779 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28721b2-9298-4ba7-8c82-c0fcc430f5ca None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:36.035651 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e80c27-7887-4c45-90eb-3786e8e58d19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:36.036240 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e80c27-7887-4c45-90eb-3786e8e58d19 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:36.036516 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e80c27-7887-4c45-90eb-3786e8e58d19 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:36.036784 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e80c27-7887-4c45-90eb-3786e8e58d19 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:36.104254 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e80c27-7887-4c45-90eb-3786e8e58d19 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:36.105670 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e80c27-7887-4c45-90eb-3786e8e58d19 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:36.133724 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:36.200303 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 556/1115] 10.222.0.16 () {58 vars in 1282 bytes} [Wed Aug 24 09:09:35 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 09:09:36.206542 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a9860a-7767-4ba6-8f2b-91f158d1efbc None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:36.352580 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a9860a-7767-4ba6-8f2b-91f158d1efbc None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:36.385132 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500aecc1-c2fb-4cf7-8d15-f7f587bfb35d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:36.385695 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500aecc1-c2fb-4cf7-8d15-f7f587bfb35d None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:36.385977 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500aecc1-c2fb-4cf7-8d15-f7f587bfb35d None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:36.386249 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500aecc1-c2fb-4cf7-8d15-f7f587bfb35d None admin] PATH_INFO: `/v3/projects/e8ae36cb945c488682ab38083533ff07` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:36.387349 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500aecc1-c2fb-4cf7-8d15-f7f587bfb35d None admin] RBAC: Authorizing `identity:get_project(project_id=e8ae36cb945c488682ab38083533ff07)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:36.388806 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500aecc1-c2fb-4cf7-8d15-f7f587bfb35d None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:36.390466 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 560/1116] 10.222.0.16 () {56 vars in 1186 bytes} [Wed Aug 24 09:09:36 2022] GET /identity/v3/projects/e8ae36cb945c488682ab38083533ff07 => generated 311 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:09:36.396596 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e80c27-7887-4c45-90eb-3786e8e58d19 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:36.494019 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e80c27-7887-4c45-90eb-3786e8e58d19 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:36.520760 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:36.526585 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd7b971-2820-4926-a4c0-3644871fb114 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:36.527175 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd7b971-2820-4926-a4c0-3644871fb114 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:36.531784 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd7b971-2820-4926-a4c0-3644871fb114 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:36.531784 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd7b971-2820-4926-a4c0-3644871fb114 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:36.531784 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd7b971-2820-4926-a4c0-3644871fb114 None admin] RBAC: Adding query filter params (role.id=ed6466d17fb7403c8491bb7f1ef95fc1, scope.project.id=e8ae36cb945c488682ab38083533ff07, user.id=1bcb46d61e2a491a880e14d7c5cfde7d) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:09:36.531784 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd7b971-2820-4926-a4c0-3644871fb114 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:36.531784 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd7b971-2820-4926-a4c0-3644871fb114 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:36.535198 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 557/1117] 10.222.0.16 () {56 vars in 1374 bytes} [Wed Aug 24 09:09:36 2022] GET /identity/v3/role_assignments?user.id=1bcb46d61e2a491a880e14d7c5cfde7d&scope.project.id=e8ae36cb945c488682ab38083533ff07&role.id=ed6466d17fb7403c8491bb7f1ef95fc1 => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:09:38.389593 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101ed88b-ddff-4167-a8b3-c2c846b4a57d None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:38.390138 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101ed88b-ddff-4167-a8b3-c2c846b4a57d None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:38.390138 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101ed88b-ddff-4167-a8b3-c2c846b4a57d None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:38.391460 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 561/1118] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:09:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:09:38.396799 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b807a78-06c0-4058-85be-26838621ef29 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:38.397213 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b807a78-06c0-4058-85be-26838621ef29 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:38.397349 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b807a78-06c0-4058-85be-26838621ef29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:38.416120 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-1b807a78-06c0-4058-85be-26838621ef29 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:09:38.417299 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-1b807a78-06c0-4058-85be-26838621ef29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:09:38.481386 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 558/1119] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:09:38 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 09:09:38.753522 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101ed88b-ddff-4167-a8b3-c2c846b4a57d None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:38.816556 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:38.857973 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101ed88b-ddff-4167-a8b3-c2c846b4a57d None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:38.892520 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c510ac3-e857-4618-8b47-fceeae6c0ce4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:38.893142 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c510ac3-e857-4618-8b47-fceeae6c0ce4 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:38.893424 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c510ac3-e857-4618-8b47-fceeae6c0ce4 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:38.893698 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c510ac3-e857-4618-8b47-fceeae6c0ce4 None admin] PATH_INFO: `/v3/roles/ed6466d17fb7403c8491bb7f1ef95fc1` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:38.895083 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c510ac3-e857-4618-8b47-fceeae6c0ce4 None admin] RBAC: Authorizing `identity:get_role(role_id=ed6466d17fb7403c8491bb7f1ef95fc1)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:38.896483 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c510ac3-e857-4618-8b47-fceeae6c0ce4 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:38.897802 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 562/1120] 10.222.0.16 () {56 vars in 1177 bytes} [Wed Aug 24 09:09:38 2022] GET /identity/v3/roles/ed6466d17fb7403c8491bb7f1ef95fc1 => generated 223 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:09:38.903728 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b807a78-06c0-4058-85be-26838621ef29 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:39.002433 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b807a78-06c0-4058-85be-26838621ef29 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:39.026148 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:39.036410 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e93d747-ee87-4eb5-b0ff-77241ebe0e6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:39.037065 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e93d747-ee87-4eb5-b0ff-77241ebe0e6f None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:39.037382 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e93d747-ee87-4eb5-b0ff-77241ebe0e6f None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:39.037660 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e93d747-ee87-4eb5-b0ff-77241ebe0e6f None admin] PATH_INFO: `/v3/users/1bcb46d61e2a491a880e14d7c5cfde7d` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:39.039344 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e93d747-ee87-4eb5-b0ff-77241ebe0e6f None admin] RBAC: Authorizing `identity:get_user(user_id=1bcb46d61e2a491a880e14d7c5cfde7d)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:39.040892 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e93d747-ee87-4eb5-b0ff-77241ebe0e6f None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:39.052801 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 559/1121] 10.222.0.16 () {56 vars in 1177 bytes} [Wed Aug 24 09:09:38 2022] GET /identity/v3/users/1bcb46d61e2a491a880e14d7c5cfde7d => generated 286 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:09:39.052925 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c510ac3-e857-4618-8b47-fceeae6c0ce4 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:39.165518 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c510ac3-e857-4618-8b47-fceeae6c0ce4 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:39.190440 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:39.198729 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0599c9cb-baf5-4206-96b5-2b7b4f18e7bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:39.201330 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0599c9cb-baf5-4206-96b5-2b7b4f18e7bc None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:39.201330 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0599c9cb-baf5-4206-96b5-2b7b4f18e7bc None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:39.201330 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0599c9cb-baf5-4206-96b5-2b7b4f18e7bc None admin] PATH_INFO: `/v3/projects/e8ae36cb945c488682ab38083533ff07` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:39.201330 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0599c9cb-baf5-4206-96b5-2b7b4f18e7bc None admin] RBAC: Authorizing `identity:get_project(project_id=e8ae36cb945c488682ab38083533ff07)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:39.202573 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0599c9cb-baf5-4206-96b5-2b7b4f18e7bc None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:39.204503 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 563/1122] 10.222.0.16 () {56 vars in 1186 bytes} [Wed Aug 24 09:09:39 2022] GET /identity/v3/projects/e8ae36cb945c488682ab38083533ff07 => generated 311 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:09:39.210448 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e93d747-ee87-4eb5-b0ff-77241ebe0e6f None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:39.305872 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e93d747-ee87-4eb5-b0ff-77241ebe0e6f None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:39.339767 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15186719-bb5e-4bfc-94fa-e563145b2926 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:39.340421 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15186719-bb5e-4bfc-94fa-e563145b2926 None admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:39.340700 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15186719-bb5e-4bfc-94fa-e563145b2926 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:39.340971 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15186719-bb5e-4bfc-94fa-e563145b2926 None admin] PATH_INFO: `/v3/projects/e8ae36cb945c488682ab38083533ff07/users/1bcb46d61e2a491a880e14d7c5cfde7d/roles/ed6466d17fb7403c8491bb7f1ef95fc1` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:39.343106 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15186719-bb5e-4bfc-94fa-e563145b2926 None admin] RBAC: Authorizing `identity:create_grant(project_id=e8ae36cb945c488682ab38083533ff07, user_id=1bcb46d61e2a491a880e14d7c5cfde7d, role_id=ed6466d17fb7403c8491bb7f1ef95fc1)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:39.344564 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15186719-bb5e-4bfc-94fa-e563145b2926 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:39.416439 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 560/1123] 10.222.0.16 () {58 vars in 1439 bytes} [Wed Aug 24 09:09:39 2022] PUT /identity/v3/projects/e8ae36cb945c488682ab38083533ff07/users/1bcb46d61e2a491a880e14d7c5cfde7d/roles/ed6466d17fb7403c8491bb7f1ef95fc1 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:09:41.291663 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd47d8a-13c7-4f64-9d3f-06563d2dc1fc None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:41.292227 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd47d8a-13c7-4f64-9d3f-06563d2dc1fc None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:41.292227 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd47d8a-13c7-4f64-9d3f-06563d2dc1fc None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:41.293564 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 564/1124] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:09:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:09:41.299191 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e85d201-6841-4f94-912f-c341a24f99ef None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:41.299670 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e85d201-6841-4f94-912f-c341a24f99ef None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:41.299776 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e85d201-6841-4f94-912f-c341a24f99ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:41.318098 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-0e85d201-6841-4f94-912f-c341a24f99ef None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:09:41.319289 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-0e85d201-6841-4f94-912f-c341a24f99ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:09:41.365501 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:41.385363 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 561/1125] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:09:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 09:09:41.661863 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd47d8a-13c7-4f64-9d3f-06563d2dc1fc None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:41.762191 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd47d8a-13c7-4f64-9d3f-06563d2dc1fc None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:41.795710 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc8a0e3-5329-45c5-8973-3b8d7ca6b892 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:41.796329 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc8a0e3-5329-45c5-8973-3b8d7ca6b892 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:41.796610 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc8a0e3-5329-45c5-8973-3b8d7ca6b892 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:41.796882 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc8a0e3-5329-45c5-8973-3b8d7ca6b892 None admin] PATH_INFO: `/v3/roles/ed6466d17fb7403c8491bb7f1ef95fc1` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:41.798269 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc8a0e3-5329-45c5-8973-3b8d7ca6b892 None admin] RBAC: Authorizing `identity:get_role(role_id=ed6466d17fb7403c8491bb7f1ef95fc1)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:41.799653 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc8a0e3-5329-45c5-8973-3b8d7ca6b892 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:41.800959 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 565/1126] 10.222.0.16 () {56 vars in 1177 bytes} [Wed Aug 24 09:09:41 2022] GET /identity/v3/roles/ed6466d17fb7403c8491bb7f1ef95fc1 => generated 223 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:09:41.806760 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e85d201-6841-4f94-912f-c341a24f99ef None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:41.901623 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e85d201-6841-4f94-912f-c341a24f99ef None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:41.934831 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00a5ef1-1595-49e2-b35c-56ac7a3bae6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:41.935389 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00a5ef1-1595-49e2-b35c-56ac7a3bae6f None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:41.935703 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00a5ef1-1595-49e2-b35c-56ac7a3bae6f None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:41.935978 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00a5ef1-1595-49e2-b35c-56ac7a3bae6f None admin] PATH_INFO: `/v3/users/1bcb46d61e2a491a880e14d7c5cfde7d` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:41.937078 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00a5ef1-1595-49e2-b35c-56ac7a3bae6f None admin] RBAC: Authorizing `identity:get_user(user_id=1bcb46d61e2a491a880e14d7c5cfde7d)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:41.938499 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00a5ef1-1595-49e2-b35c-56ac7a3bae6f None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:41.940078 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 562/1127] 10.222.0.16 () {56 vars in 1177 bytes} [Wed Aug 24 09:09:41 2022] GET /identity/v3/users/1bcb46d61e2a491a880e14d7c5cfde7d => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:09:41.945616 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc8a0e3-5329-45c5-8973-3b8d7ca6b892 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:42.010815 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:42.048443 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc8a0e3-5329-45c5-8973-3b8d7ca6b892 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:42.081810 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947d672c-07c8-4d9e-a9e3-16c0624097e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:42.082411 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947d672c-07c8-4d9e-a9e3-16c0624097e6 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:42.082694 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947d672c-07c8-4d9e-a9e3-16c0624097e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:42.082974 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947d672c-07c8-4d9e-a9e3-16c0624097e6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:42.147775 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947d672c-07c8-4d9e-a9e3-16c0624097e6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:42.149211 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947d672c-07c8-4d9e-a9e3-16c0624097e6 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:42.206783 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:42.245069 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 566/1128] 10.222.0.16 () {58 vars in 1282 bytes} [Wed Aug 24 09:09:41 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 24 09:09:42.250950 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00a5ef1-1595-49e2-b35c-56ac7a3bae6f None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:42.299550 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:42.348107 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00a5ef1-1595-49e2-b35c-56ac7a3bae6f None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:42.381272 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5b9a99-50ec-49b0-b0dc-8bcc008c6749 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:42.381891 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5b9a99-50ec-49b0-b0dc-8bcc008c6749 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:42.382171 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5b9a99-50ec-49b0-b0dc-8bcc008c6749 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:42.382446 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5b9a99-50ec-49b0-b0dc-8bcc008c6749 None admin] PATH_INFO: `/v3/projects/e8ae36cb945c488682ab38083533ff07` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:42.383554 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5b9a99-50ec-49b0-b0dc-8bcc008c6749 None admin] RBAC: Authorizing `identity:get_project(project_id=e8ae36cb945c488682ab38083533ff07)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:42.385022 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5b9a99-50ec-49b0-b0dc-8bcc008c6749 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:42.386647 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 563/1129] 10.222.0.16 () {56 vars in 1186 bytes} [Wed Aug 24 09:09:42 2022] GET /identity/v3/projects/e8ae36cb945c488682ab38083533ff07 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:09:42.392830 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947d672c-07c8-4d9e-a9e3-16c0624097e6 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:42.489804 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947d672c-07c8-4d9e-a9e3-16c0624097e6 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:42.522034 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec92a4c-3805-46d6-843c-7747673a025d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:42.523140 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec92a4c-3805-46d6-843c-7747673a025d None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:42.523416 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec92a4c-3805-46d6-843c-7747673a025d None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:42.523714 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec92a4c-3805-46d6-843c-7747673a025d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:42.524426 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec92a4c-3805-46d6-843c-7747673a025d None admin] RBAC: Adding query filter params (role.id=ed6466d17fb7403c8491bb7f1ef95fc1, scope.project.id=e8ae36cb945c488682ab38083533ff07, user.id=1bcb46d61e2a491a880e14d7c5cfde7d) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:09:42.524728 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec92a4c-3805-46d6-843c-7747673a025d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:42.526132 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec92a4c-3805-46d6-843c-7747673a025d None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:42.531711 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 567/1130] 10.222.0.16 () {56 vars in 1374 bytes} [Wed Aug 24 09:09:42 2022] GET /identity/v3/role_assignments?user.id=1bcb46d61e2a491a880e14d7c5cfde7d&scope.project.id=e8ae36cb945c488682ab38083533ff07&role.id=ed6466d17fb7403c8491bb7f1ef95fc1 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:09:44.421828 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933b8642-1070-48ab-a2ba-a429bc4bad34 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:44.422382 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933b8642-1070-48ab-a2ba-a429bc4bad34 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:44.422382 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933b8642-1070-48ab-a2ba-a429bc4bad34 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:44.423757 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 564/1131] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:09:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:09:44.429798 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed14f9d0-635e-43ca-8b1b-a3f1483297b2 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:44.430237 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed14f9d0-635e-43ca-8b1b-a3f1483297b2 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:44.430349 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed14f9d0-635e-43ca-8b1b-a3f1483297b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:44.448683 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-ed14f9d0-635e-43ca-8b1b-a3f1483297b2 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:09:44.449864 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-ed14f9d0-635e-43ca-8b1b-a3f1483297b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:09:44.469269 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:44.514596 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 568/1132] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:09:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 09:09:44.780985 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933b8642-1070-48ab-a2ba-a429bc4bad34 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:44.879767 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933b8642-1070-48ab-a2ba-a429bc4bad34 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:44.896720 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:44.912718 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36865891-fee4-48de-bc8e-6931bd4c15af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:44.913863 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36865891-fee4-48de-bc8e-6931bd4c15af None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:44.914147 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36865891-fee4-48de-bc8e-6931bd4c15af None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:44.914423 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36865891-fee4-48de-bc8e-6931bd4c15af None admin] PATH_INFO: `/v3/domains/default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:44.915538 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36865891-fee4-48de-bc8e-6931bd4c15af None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:44.916989 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36865891-fee4-48de-bc8e-6931bd4c15af None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:44.918588 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 565/1133] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 24 09:09:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:09:44.924342 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed14f9d0-635e-43ca-8b1b-a3f1483297b2 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:45.020057 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed14f9d0-635e-43ca-8b1b-a3f1483297b2 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:45.052530 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c77f6f-de07-49bd-8835-3e4774b6978d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:45.053075 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c77f6f-de07-49bd-8835-3e4774b6978d None admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:45.053355 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c77f6f-de07-49bd-8835-3e4774b6978d None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:45.053645 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c77f6f-de07-49bd-8835-3e4774b6978d None admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:45.054821 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c77f6f-de07-49bd-8835-3e4774b6978d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:45.056267 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c77f6f-de07-49bd-8835-3e4774b6978d None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:45.157613 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 569/1134] 10.222.0.16 () {60 vars in 1141 bytes} [Wed Aug 24 09:09:44 2022] POST /identity/v3/projects => generated 311 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:09:47.113150 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d3a3f3-c9ac-4fff-872e-a248abcd4a2b None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:47.113691 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d3a3f3-c9ac-4fff-872e-a248abcd4a2b None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:47.113691 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d3a3f3-c9ac-4fff-872e-a248abcd4a2b None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:47.115029 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 566/1135] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:09:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 09:09:47.120572 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7630789f-3a93-4c2a-8cf4-1e0b705c322f None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:47.121047 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7630789f-3a93-4c2a-8cf4-1e0b705c322f None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:47.121141 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7630789f-3a93-4c2a-8cf4-1e0b705c322f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:47.139406 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-7630789f-3a93-4c2a-8cf4-1e0b705c322f None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:09:47.140613 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-7630789f-3a93-4c2a-8cf4-1e0b705c322f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:09:47.142556 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:47.206403 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 570/1136] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:09:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 09:09:47.477119 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d3a3f3-c9ac-4fff-872e-a248abcd4a2b None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:47.577258 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d3a3f3-c9ac-4fff-872e-a248abcd4a2b None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:47.618394 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b3d2d8-c85c-42c6-9c16-c20966c663a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:47.619014 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b3d2d8-c85c-42c6-9c16-c20966c663a9 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:47.619294 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b3d2d8-c85c-42c6-9c16-c20966c663a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:47.619570 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b3d2d8-c85c-42c6-9c16-c20966c663a9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:47.620784 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b3d2d8-c85c-42c6-9c16-c20966c663a9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:47.622211 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b3d2d8-c85c-42c6-9c16-c20966c663a9 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:47.623812 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 567/1137] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 24 09:09:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:09:47.629648 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7630789f-3a93-4c2a-8cf4-1e0b705c322f None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:47.727428 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7630789f-3a93-4c2a-8cf4-1e0b705c322f None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:47.759167 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe27148b-2e43-4a0c-8783-1522ffedca17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:47.759869 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe27148b-2e43-4a0c-8783-1522ffedca17 None admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:47.760150 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe27148b-2e43-4a0c-8783-1522ffedca17 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:47.760420 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe27148b-2e43-4a0c-8783-1522ffedca17 None admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:47.761056 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe27148b-2e43-4a0c-8783-1522ffedca17 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:47.762473 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe27148b-2e43-4a0c-8783-1522ffedca17 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:47.772060 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:47.861571 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 571/1138] 10.222.0.16 () {60 vars in 1132 bytes} [Wed Aug 24 09:09:47 2022] POST /identity/v3/users => generated 286 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:09:49.813920 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac98b57-8684-4bf3-8aa0-52f87d0423d9 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:49.814433 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac98b57-8684-4bf3-8aa0-52f87d0423d9 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:49.814544 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac98b57-8684-4bf3-8aa0-52f87d0423d9 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:49.815881 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 568/1139] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:09:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:09:49.821696 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a745428e-061a-4451-9ea5-549b7cf778c5 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:49.822122 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a745428e-061a-4451-9ea5-549b7cf778c5 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:49.822216 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a745428e-061a-4451-9ea5-549b7cf778c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:49.840162 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-a745428e-061a-4451-9ea5-549b7cf778c5 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:09:49.841351 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-a745428e-061a-4451-9ea5-549b7cf778c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:09:49.907306 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 572/1140] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:09:49 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 09:09:50.172778 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac98b57-8684-4bf3-8aa0-52f87d0423d9 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:50.231731 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:50.276690 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac98b57-8684-4bf3-8aa0-52f87d0423d9 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:50.311370 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dca5aee-b18c-4d11-bdb6-013443532a40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:50.312067 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dca5aee-b18c-4d11-bdb6-013443532a40 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:50.312341 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dca5aee-b18c-4d11-bdb6-013443532a40 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:50.312612 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dca5aee-b18c-4d11-bdb6-013443532a40 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:50.322448 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dca5aee-b18c-4d11-bdb6-013443532a40 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:09:50.327084 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-2dca5aee-b18c-4d11-bdb6-013443532a40 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:09:50.327084 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:09:50.327084 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:09:50.327084 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:09:50.327084 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:09:50.327084 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:09:50.327084 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:09:50.327084 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:09:50.327084 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:09:50.327084 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:09:50.327084 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:09:50.327084 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:09:50.327084 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:09:50.327084 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:09:50.327084 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:09:50.327084 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:09:50.328436 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:09:50.328436 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:09:50.328436 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:09:50.328436 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:09:50.328436 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:09:50.328436 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:09:50.328436 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:09:50.328436 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:09:50.328436 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:09:50.328436 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:09:50.328436 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:09:50.328436 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 24 09:09:50.328436 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:09:50.328436 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:09:50.328436 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:09:50.328436 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:09:50.329389 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:09:50.329389 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:09:50.329389 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:09:50.329389 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:09:50.329389 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:09:50.329389 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:09:50.329389 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:09:50.329389 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:09:50.329389 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:09:50.329389 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:09:50.329389 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:09:50.329389 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 24 09:09:50.329389 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 569/1141] 10.222.0.16 () {56 vars in 1096 bytes} [Wed Aug 24 09:09:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 09:09:50.334528 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a745428e-061a-4451-9ea5-549b7cf778c5 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:50.432180 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a745428e-061a-4451-9ea5-549b7cf778c5 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:50.454828 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:50.465275 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f203b404-54c4-4ff7-aadb-3e15c699fc77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:50.465882 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f203b404-54c4-4ff7-aadb-3e15c699fc77 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:50.466163 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f203b404-54c4-4ff7-aadb-3e15c699fc77 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:50.466435 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f203b404-54c4-4ff7-aadb-3e15c699fc77 None admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:50.467101 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f203b404-54c4-4ff7-aadb-3e15c699fc77 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:09:50.467403 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f203b404-54c4-4ff7-aadb-3e15c699fc77 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:50.468793 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f203b404-54c4-4ff7-aadb-3e15c699fc77 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:50.477542 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 573/1142] 10.222.0.16 () {56 vars in 1099 bytes} [Wed Aug 24 09:09:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:09:50.483324 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dca5aee-b18c-4d11-bdb6-013443532a40 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:50.581533 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dca5aee-b18c-4d11-bdb6-013443532a40 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:50.619701 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:50.638494 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f766f7f-dcb5-4f3c-b06a-f595d136c45e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:50.639111 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f766f7f-dcb5-4f3c-b06a-f595d136c45e None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:50.639385 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f766f7f-dcb5-4f3c-b06a-f595d136c45e None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:50.639683 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f766f7f-dcb5-4f3c-b06a-f595d136c45e None admin] PATH_INFO: `/v3/users/04c97158fb384e1c8f44742c992c858e` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:50.652832 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f766f7f-dcb5-4f3c-b06a-f595d136c45e None admin] RBAC: Authorizing `identity:get_user(user_id=04c97158fb384e1c8f44742c992c858e)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:50.654339 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f766f7f-dcb5-4f3c-b06a-f595d136c45e None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:50.655915 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 570/1143] 10.222.0.16 () {56 vars in 1177 bytes} [Wed Aug 24 09:09:50 2022] GET /identity/v3/users/04c97158fb384e1c8f44742c992c858e => generated 286 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:09:50.661894 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f203b404-54c4-4ff7-aadb-3e15c699fc77 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:50.822338 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f203b404-54c4-4ff7-aadb-3e15c699fc77 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:50.855348 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1c30c1-9900-403d-9ac5-66f09037c579 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:50.855966 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1c30c1-9900-403d-9ac5-66f09037c579 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:50.856245 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1c30c1-9900-403d-9ac5-66f09037c579 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:50.856515 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1c30c1-9900-403d-9ac5-66f09037c579 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:50.908362 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:50.921189 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1c30c1-9900-403d-9ac5-66f09037c579 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:50.922626 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1c30c1-9900-403d-9ac5-66f09037c579 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:51.016338 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 574/1144] 10.222.0.16 () {58 vars in 1282 bytes} [Wed Aug 24 09:09:50 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 09:09:51.022283 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f766f7f-dcb5-4f3c-b06a-f595d136c45e None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:51.118181 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f766f7f-dcb5-4f3c-b06a-f595d136c45e None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:51.149986 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c80b6ce-42ec-4658-b426-3169c2568859 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:51.150598 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c80b6ce-42ec-4658-b426-3169c2568859 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:51.150878 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c80b6ce-42ec-4658-b426-3169c2568859 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:51.151151 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c80b6ce-42ec-4658-b426-3169c2568859 None admin] PATH_INFO: `/v3/projects/c4be5c6dc8ed490f947c13a2e23b9984` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:51.152264 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c80b6ce-42ec-4658-b426-3169c2568859 None admin] RBAC: Authorizing `identity:get_project(project_id=c4be5c6dc8ed490f947c13a2e23b9984)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:51.153678 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c80b6ce-42ec-4658-b426-3169c2568859 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:51.155290 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 571/1145] 10.222.0.16 () {56 vars in 1186 bytes} [Wed Aug 24 09:09:51 2022] GET /identity/v3/projects/c4be5c6dc8ed490f947c13a2e23b9984 => generated 311 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:09:51.161296 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1c30c1-9900-403d-9ac5-66f09037c579 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:51.243506 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:51.257638 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1c30c1-9900-403d-9ac5-66f09037c579 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:51.290763 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bf8414-cf9e-4014-a5fe-8c148537a2e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:51.291357 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bf8414-cf9e-4014-a5fe-8c148537a2e1 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:51.291657 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bf8414-cf9e-4014-a5fe-8c148537a2e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:51.291929 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bf8414-cf9e-4014-a5fe-8c148537a2e1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:51.292639 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bf8414-cf9e-4014-a5fe-8c148537a2e1 None admin] RBAC: Adding query filter params (role.id=15f634d4a2d247ddab90c458e5e1c141, scope.project.id=c4be5c6dc8ed490f947c13a2e23b9984, user.id=04c97158fb384e1c8f44742c992c858e) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:09:51.292941 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bf8414-cf9e-4014-a5fe-8c148537a2e1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:51.294355 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bf8414-cf9e-4014-a5fe-8c148537a2e1 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:51.299654 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 575/1146] 10.222.0.16 () {56 vars in 1374 bytes} [Wed Aug 24 09:09:51 2022] GET /identity/v3/role_assignments?user.id=04c97158fb384e1c8f44742c992c858e&scope.project.id=c4be5c6dc8ed490f947c13a2e23b9984&role.id=15f634d4a2d247ddab90c458e5e1c141 => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:09:53.152625 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72047476-6b3c-4e61-b96c-a623cf2d6de8 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:53.153178 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72047476-6b3c-4e61-b96c-a623cf2d6de8 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:53.153178 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72047476-6b3c-4e61-b96c-a623cf2d6de8 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:53.154497 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 572/1147] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:09:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:09:53.159537 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c496d1b7-74ab-465a-94d9-cf691a1ca3f3 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:53.159982 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c496d1b7-74ab-465a-94d9-cf691a1ca3f3 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:53.160258 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c496d1b7-74ab-465a-94d9-cf691a1ca3f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:53.177785 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-c496d1b7-74ab-465a-94d9-cf691a1ca3f3 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:09:53.178974 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-c496d1b7-74ab-465a-94d9-cf691a1ca3f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:09:53.242446 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 576/1148] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:09:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 09:09:53.515388 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72047476-6b3c-4e61-b96c-a623cf2d6de8 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:53.557851 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:53.614563 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72047476-6b3c-4e61-b96c-a623cf2d6de8 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:53.646593 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b540bfc2-d69b-44f7-8a7b-06dc4f3b0519 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:53.647768 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b540bfc2-d69b-44f7-8a7b-06dc4f3b0519 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:53.648049 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b540bfc2-d69b-44f7-8a7b-06dc4f3b0519 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:53.648322 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b540bfc2-d69b-44f7-8a7b-06dc4f3b0519 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:53.658498 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b540bfc2-d69b-44f7-8a7b-06dc4f3b0519 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:09:53.664308 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-b540bfc2-d69b-44f7-8a7b-06dc4f3b0519 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:09:53.664308 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:09:53.664308 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:09:53.664308 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:09:53.664308 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:09:53.664308 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:09:53.664308 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:09:53.664308 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:09:53.664308 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:09:53.664308 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:09:53.664308 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:09:53.664308 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:09:53.664308 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:09:53.664308 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:09:53.664308 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:09:53.664308 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:09:53.665426 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:09:53.665426 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:09:53.665426 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:09:53.665426 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:09:53.665426 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:09:53.665426 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:09:53.665426 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:09:53.665426 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:09:53.665426 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:09:53.665426 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:09:53.665426 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:09:53.665426 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 24 09:09:53.665426 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:09:53.665426 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:09:53.665426 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:09:53.665426 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:09:53.666310 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:09:53.666310 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:09:53.666310 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:09:53.666310 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:09:53.666310 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:09:53.666310 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:09:53.666310 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:09:53.666310 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:09:53.666310 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:09:53.666310 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:09:53.666310 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:09:53.666310 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 24 09:09:53.666310 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 573/1149] 10.222.0.16 () {56 vars in 1096 bytes} [Wed Aug 24 09:09:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:09:53.671435 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c496d1b7-74ab-465a-94d9-cf691a1ca3f3 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:53.752401 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:53.770880 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c496d1b7-74ab-465a-94d9-cf691a1ca3f3 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:53.805418 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a99fe9d-64c6-4bef-aa76-1a550b0ba011 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:53.806018 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a99fe9d-64c6-4bef-aa76-1a550b0ba011 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:53.806302 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a99fe9d-64c6-4bef-aa76-1a550b0ba011 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:53.806577 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a99fe9d-64c6-4bef-aa76-1a550b0ba011 None admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:53.807244 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a99fe9d-64c6-4bef-aa76-1a550b0ba011 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:09:53.807551 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a99fe9d-64c6-4bef-aa76-1a550b0ba011 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:53.808944 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a99fe9d-64c6-4bef-aa76-1a550b0ba011 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:53.816790 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 577/1150] 10.222.0.16 () {56 vars in 1099 bytes} [Wed Aug 24 09:09:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:09:53.822641 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b540bfc2-d69b-44f7-8a7b-06dc4f3b0519 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:53.917346 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:53.920144 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b540bfc2-d69b-44f7-8a7b-06dc4f3b0519 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:53.952750 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653ecefe-ade9-4e70-9cd2-e55c2a70b8f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:53.953830 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653ecefe-ade9-4e70-9cd2-e55c2a70b8f7 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:53.954132 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653ecefe-ade9-4e70-9cd2-e55c2a70b8f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:53.954741 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653ecefe-ade9-4e70-9cd2-e55c2a70b8f7 None admin] PATH_INFO: `/v3/users/04c97158fb384e1c8f44742c992c858e` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:53.955923 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653ecefe-ade9-4e70-9cd2-e55c2a70b8f7 None admin] RBAC: Authorizing `identity:get_user(user_id=04c97158fb384e1c8f44742c992c858e)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:53.957397 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653ecefe-ade9-4e70-9cd2-e55c2a70b8f7 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:53.958961 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 574/1151] 10.222.0.16 () {56 vars in 1177 bytes} [Wed Aug 24 09:09:53 2022] GET /identity/v3/users/04c97158fb384e1c8f44742c992c858e => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:09:53.964735 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a99fe9d-64c6-4bef-aa76-1a550b0ba011 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:54.062450 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a99fe9d-64c6-4bef-aa76-1a550b0ba011 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:54.094405 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321392c2-3096-4e8b-b414-2987505df09d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:54.095023 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321392c2-3096-4e8b-b414-2987505df09d None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:54.095312 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321392c2-3096-4e8b-b414-2987505df09d None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:54.095577 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321392c2-3096-4e8b-b414-2987505df09d None admin] PATH_INFO: `/v3/projects/c4be5c6dc8ed490f947c13a2e23b9984` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:54.096710 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321392c2-3096-4e8b-b414-2987505df09d None admin] RBAC: Authorizing `identity:get_project(project_id=c4be5c6dc8ed490f947c13a2e23b9984)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:54.098134 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321392c2-3096-4e8b-b414-2987505df09d None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:54.099756 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 578/1152] 10.222.0.16 () {56 vars in 1186 bytes} [Wed Aug 24 09:09:53 2022] GET /identity/v3/projects/c4be5c6dc8ed490f947c13a2e23b9984 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:09:54.105970 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653ecefe-ade9-4e70-9cd2-e55c2a70b8f7 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:54.201696 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653ecefe-ade9-4e70-9cd2-e55c2a70b8f7 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:54.235141 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e4942e-522b-4fa4-b83e-e26db29608a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:54.235808 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e4942e-522b-4fa4-b83e-e26db29608a6 None admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:54.236082 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e4942e-522b-4fa4-b83e-e26db29608a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:54.236349 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e4942e-522b-4fa4-b83e-e26db29608a6 None admin] PATH_INFO: `/v3/projects/c4be5c6dc8ed490f947c13a2e23b9984/users/04c97158fb384e1c8f44742c992c858e/roles/15f634d4a2d247ddab90c458e5e1c141` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:54.238193 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e4942e-522b-4fa4-b83e-e26db29608a6 None admin] RBAC: Authorizing `identity:create_grant(project_id=c4be5c6dc8ed490f947c13a2e23b9984, user_id=04c97158fb384e1c8f44742c992c858e, role_id=15f634d4a2d247ddab90c458e5e1c141)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:54.239600 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e4942e-522b-4fa4-b83e-e26db29608a6 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:54.353636 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 575/1153] 10.222.0.16 () {58 vars in 1439 bytes} [Wed Aug 24 09:09:54 2022] PUT /identity/v3/projects/c4be5c6dc8ed490f947c13a2e23b9984/users/04c97158fb384e1c8f44742c992c858e/roles/15f634d4a2d247ddab90c458e5e1c141 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:09:56.233279 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0681420b-39c7-44c2-bad4-d6b0fcd7ee13 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:56.233836 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0681420b-39c7-44c2-bad4-d6b0fcd7ee13 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:56.233836 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0681420b-39c7-44c2-bad4-d6b0fcd7ee13 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:56.235150 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 579/1154] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:09:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:09:56.240327 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa44ae58-2439-4657-9b00-db59c159afc5 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:56.241384 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa44ae58-2439-4657-9b00-db59c159afc5 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:56.241476 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa44ae58-2439-4657-9b00-db59c159afc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:56.259569 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-fa44ae58-2439-4657-9b00-db59c159afc5 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:09:56.260776 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-fa44ae58-2439-4657-9b00-db59c159afc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:09:56.283243 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:56.326194 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 576/1155] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:09:56 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 09:09:56.596360 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0681420b-39c7-44c2-bad4-d6b0fcd7ee13 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:56.609572 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:56.695110 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0681420b-39c7-44c2-bad4-d6b0fcd7ee13 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:56.728200 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6103fcc4-f2db-44f2-b304-733a58ff3005 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:56.728831 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6103fcc4-f2db-44f2-b304-733a58ff3005 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:56.732961 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6103fcc4-f2db-44f2-b304-733a58ff3005 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:56.732961 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6103fcc4-f2db-44f2-b304-733a58ff3005 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:56.739909 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6103fcc4-f2db-44f2-b304-733a58ff3005 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:09:56.744569 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-6103fcc4-f2db-44f2-b304-733a58ff3005 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:09:56.744569 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:09:56.744569 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:09:56.744569 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:09:56.744569 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:09:56.744569 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:09:56.744569 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:09:56.744569 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:09:56.744569 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:09:56.744569 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:09:56.744569 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:09:56.744569 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:09:56.744569 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:09:56.744569 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:09:56.744569 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:09:56.744569 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:09:56.745750 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:09:56.745750 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:09:56.745750 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:09:56.745750 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:09:56.745750 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:09:56.745750 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:09:56.745750 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:09:56.745750 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:09:56.745750 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:09:56.745750 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:09:56.745750 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:09:56.745750 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Aug 24 09:09:56.745750 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:09:56.745750 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:09:56.745750 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:09:56.745750 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:09:56.746958 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:09:56.746958 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:09:56.746958 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:09:56.746958 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:09:56.746958 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:09:56.746958 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:09:56.746958 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:09:56.746958 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:09:56.746958 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:09:56.746958 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:09:56.746958 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:09:56.746958 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Aug 24 09:09:56.746958 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 580/1156] 10.222.0.16 () {56 vars in 1096 bytes} [Wed Aug 24 09:09:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:09:56.751644 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa44ae58-2439-4657-9b00-db59c159afc5 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:56.846904 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa44ae58-2439-4657-9b00-db59c159afc5 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:56.879753 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a8bb7e-aeb4-45fd-a808-06204f7111e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:56.880349 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a8bb7e-aeb4-45fd-a808-06204f7111e6 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:56.880626 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a8bb7e-aeb4-45fd-a808-06204f7111e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:56.880895 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a8bb7e-aeb4-45fd-a808-06204f7111e6 None admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:56.881554 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a8bb7e-aeb4-45fd-a808-06204f7111e6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:09:56.881857 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a8bb7e-aeb4-45fd-a808-06204f7111e6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:56.883760 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a8bb7e-aeb4-45fd-a808-06204f7111e6 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:56.891434 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 577/1157] 10.222.0.16 () {56 vars in 1099 bytes} [Wed Aug 24 09:09:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:09:56.897195 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6103fcc4-f2db-44f2-b304-733a58ff3005 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:56.965117 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:56.995352 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6103fcc4-f2db-44f2-b304-733a58ff3005 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:57.027254 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af9e8ce-47bd-4500-a094-5241eeb6ec76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:57.027897 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af9e8ce-47bd-4500-a094-5241eeb6ec76 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:57.028174 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af9e8ce-47bd-4500-a094-5241eeb6ec76 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:57.028444 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af9e8ce-47bd-4500-a094-5241eeb6ec76 None admin] PATH_INFO: `/v3/users/04c97158fb384e1c8f44742c992c858e` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:57.029543 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af9e8ce-47bd-4500-a094-5241eeb6ec76 None admin] RBAC: Authorizing `identity:get_user(user_id=04c97158fb384e1c8f44742c992c858e)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:57.030961 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af9e8ce-47bd-4500-a094-5241eeb6ec76 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:57.032529 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 581/1158] 10.222.0.16 () {56 vars in 1177 bytes} [Wed Aug 24 09:09:56 2022] GET /identity/v3/users/04c97158fb384e1c8f44742c992c858e => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:09:57.038232 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a8bb7e-aeb4-45fd-a808-06204f7111e6 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:57.060690 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:57.137365 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a8bb7e-aeb4-45fd-a808-06204f7111e6 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:57.171023 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363172a3-523c-430d-a7aa-fb63d3b4d538 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:57.171746 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363172a3-523c-430d-a7aa-fb63d3b4d538 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:57.172028 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363172a3-523c-430d-a7aa-fb63d3b4d538 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:57.172293 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363172a3-523c-430d-a7aa-fb63d3b4d538 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:57.237331 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363172a3-523c-430d-a7aa-fb63d3b4d538 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:57.238758 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363172a3-523c-430d-a7aa-fb63d3b4d538 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:57.258616 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:57.333212 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 578/1159] 10.222.0.16 () {58 vars in 1282 bytes} [Wed Aug 24 09:09:57 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 09:09:57.339791 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af9e8ce-47bd-4500-a094-5241eeb6ec76 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:57.438971 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af9e8ce-47bd-4500-a094-5241eeb6ec76 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:57.472527 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd2510e-83cb-4f29-bcc7-2487d59cde49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:57.473150 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd2510e-83cb-4f29-bcc7-2487d59cde49 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:57.473426 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd2510e-83cb-4f29-bcc7-2487d59cde49 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:57.473698 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd2510e-83cb-4f29-bcc7-2487d59cde49 None admin] PATH_INFO: `/v3/projects/c4be5c6dc8ed490f947c13a2e23b9984` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:57.474796 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd2510e-83cb-4f29-bcc7-2487d59cde49 None admin] RBAC: Authorizing `identity:get_project(project_id=c4be5c6dc8ed490f947c13a2e23b9984)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:57.476259 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd2510e-83cb-4f29-bcc7-2487d59cde49 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:57.478402 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 582/1160] 10.222.0.16 () {56 vars in 1186 bytes} [Wed Aug 24 09:09:57 2022] GET /identity/v3/projects/c4be5c6dc8ed490f947c13a2e23b9984 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:09:57.484588 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363172a3-523c-430d-a7aa-fb63d3b4d538 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:57.579142 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363172a3-523c-430d-a7aa-fb63d3b4d538 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:09:57.598326 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:57.612295 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4501059c-ab91-452d-8f2e-614e049a5c06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:09:57.612888 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4501059c-ab91-452d-8f2e-614e049a5c06 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:57.613166 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4501059c-ab91-452d-8f2e-614e049a5c06 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:57.613437 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4501059c-ab91-452d-8f2e-614e049a5c06 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:57.614263 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4501059c-ab91-452d-8f2e-614e049a5c06 None admin] RBAC: Adding query filter params (role.id=15f634d4a2d247ddab90c458e5e1c141, scope.project.id=c4be5c6dc8ed490f947c13a2e23b9984, user.id=04c97158fb384e1c8f44742c992c858e) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:09:57.614574 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4501059c-ab91-452d-8f2e-614e049a5c06 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:09:57.616009 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4501059c-ab91-452d-8f2e-614e049a5c06 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:09:57.621630 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 579/1161] 10.222.0.16 () {56 vars in 1374 bytes} [Wed Aug 24 09:09:57 2022] GET /identity/v3/role_assignments?user.id=04c97158fb384e1c8f44742c992c858e&scope.project.id=c4be5c6dc8ed490f947c13a2e23b9984&role.id=15f634d4a2d247ddab90c458e5e1c141 => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:09:59.520988 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514fba2c-6e4f-49c9-aeba-86defe66db54 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:59.521542 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514fba2c-6e4f-49c9-aeba-86defe66db54 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:59.521542 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514fba2c-6e4f-49c9-aeba-86defe66db54 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:59.522847 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 583/1162] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:09:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:09:59.528587 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28aa8588-7552-4878-983e-25cc9bb7af0f None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:09:59.529033 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28aa8588-7552-4878-983e-25cc9bb7af0f None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:09:59.529139 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28aa8588-7552-4878-983e-25cc9bb7af0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:09:59.547202 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-28aa8588-7552-4878-983e-25cc9bb7af0f None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:09:59.548405 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-28aa8588-7552-4878-983e-25cc9bb7af0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:09:59.612346 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 580/1163] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:09:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 09:09:59.877526 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514fba2c-6e4f-49c9-aeba-86defe66db54 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:09:59.887265 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:09:59.976792 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514fba2c-6e4f-49c9-aeba-86defe66db54 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:10:00.009006 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28161ca-2db0-4069-9a7f-3b1ef58fc474 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:10:00.009597 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28161ca-2db0-4069-9a7f-3b1ef58fc474 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:00.009873 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28161ca-2db0-4069-9a7f-3b1ef58fc474 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:00.010144 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28161ca-2db0-4069-9a7f-3b1ef58fc474 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:00.073845 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28161ca-2db0-4069-9a7f-3b1ef58fc474 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:10:00.075271 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28161ca-2db0-4069-9a7f-3b1ef58fc474 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:10:00.082660 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:10:00.172798 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 584/1164] 10.222.0.16 () {58 vars in 1282 bytes} [Wed Aug 24 09:09:59 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 09:10:00.178675 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28aa8588-7552-4878-983e-25cc9bb7af0f None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:10:00.275131 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28aa8588-7552-4878-983e-25cc9bb7af0f None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:10:00.288557 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:10:00.308465 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34f58fd-4972-4267-929e-0f122c2bdd71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:10:00.309140 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34f58fd-4972-4267-929e-0f122c2bdd71 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:00.309423 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34f58fd-4972-4267-929e-0f122c2bdd71 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:00.309696 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34f58fd-4972-4267-929e-0f122c2bdd71 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:00.316503 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34f58fd-4972-4267-929e-0f122c2bdd71 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:10:00.317981 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34f58fd-4972-4267-929e-0f122c2bdd71 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:10:00.330038 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-b34f58fd-4972-4267-929e-0f122c2bdd71 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 24 09:10:00.330038 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:10:00.330038 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 24 09:10:00.330038 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 24 09:10:00.330038 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 09:10:00.330038 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 09:10:00.330038 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 09:10:00.330038 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 09:10:00.330038 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Aug 24 09:10:00.330038 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 24 09:10:00.330038 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 09:10:00.330038 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 24 09:10:00.330038 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:10:00.330038 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:10:00.330038 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:10:00.330038 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:10:00.330038 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:10:00.331248 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:10:00.331248 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:10:00.331248 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:10:00.331248 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:10:00.331248 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:10:00.331248 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:10:00.331248 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:10:00.331248 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:10:00.331248 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:10:00.331248 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:10:00.331248 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:10:00.331248 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:10:00.331248 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:10:00.331248 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:10:00.331248 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:10:00.331248 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:10:00.332348 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:10:00.332348 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 24 09:10:00.332348 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:10:00.332348 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:10:00.332348 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:10:00.332348 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:10:00.332348 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:10:00.332348 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:10:00.332348 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:10:00.332348 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:10:00.332348 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 24 09:10:00.332348 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:10:00.332348 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 24 09:10:00.332348 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 24 09:10:00.332348 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 581/1165] 10.222.0.16 () {56 vars in 1117 bytes} [Wed Aug 24 09:10:00 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:10:00.337770 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28161ca-2db0-4069-9a7f-3b1ef58fc474 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:10:00.433381 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28161ca-2db0-4069-9a7f-3b1ef58fc474 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:10:00.440340 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:10:00.466516 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8821dad8-4de2-4d9c-81dd-ce3e392dab7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:10:00.467110 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8821dad8-4de2-4d9c-81dd-ce3e392dab7f None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:00.467390 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8821dad8-4de2-4d9c-81dd-ce3e392dab7f None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:00.467691 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8821dad8-4de2-4d9c-81dd-ce3e392dab7f None admin] PATH_INFO: `/v3/domains` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:00.468326 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8821dad8-4de2-4d9c-81dd-ce3e392dab7f None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:10:00.468626 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8821dad8-4de2-4d9c-81dd-ce3e392dab7f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:10:00.470018 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8821dad8-4de2-4d9c-81dd-ce3e392dab7f None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:10:00.477714 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 585/1166] 10.222.0.16 () {56 vars in 1115 bytes} [Wed Aug 24 09:10:00 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:10:02.348066 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049e76e3-1b5f-473c-8352-05002c1aec91 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:02.348564 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049e76e3-1b5f-473c-8352-05002c1aec91 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:02.348677 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049e76e3-1b5f-473c-8352-05002c1aec91 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:02.349948 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 582/1167] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:10:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:10:02.355233 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80134aa4-29d4-44ce-aff6-5bd7e6e929ca None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:02.355636 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80134aa4-29d4-44ce-aff6-5bd7e6e929ca None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:02.355806 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80134aa4-29d4-44ce-aff6-5bd7e6e929ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:02.373321 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-80134aa4-29d4-44ce-aff6-5bd7e6e929ca None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:10:02.374502 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-80134aa4-29d4-44ce-aff6-5bd7e6e929ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:10:02.439219 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 586/1168] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:10:02 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 09:10:02.706615 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049e76e3-1b5f-473c-8352-05002c1aec91 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:10:02.805214 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049e76e3-1b5f-473c-8352-05002c1aec91 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:10:02.838370 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddca77eb-3602-4843-814a-947a4836e4f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:10:02.838972 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddca77eb-3602-4843-814a-947a4836e4f7 None admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:02.839255 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddca77eb-3602-4843-814a-947a4836e4f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:02.839532 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddca77eb-3602-4843-814a-947a4836e4f7 None admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:02.840227 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddca77eb-3602-4843-814a-947a4836e4f7 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:10:02.841589 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddca77eb-3602-4843-814a-947a4836e4f7 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:10:02.922251 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 583/1169] 10.222.0.16 () {60 vars in 1138 bytes} [Wed Aug 24 09:10:02 2022] POST /identity/v3/domains => generated 267 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:10:04.900639 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923f766d-c8c4-4725-88ad-14592741063d None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:04.901172 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923f766d-c8c4-4725-88ad-14592741063d None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:04.901172 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923f766d-c8c4-4725-88ad-14592741063d None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:04.902518 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 587/1170] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:10:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:10:04.907724 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853bc826-583c-4636-bd91-7c1e0bd1ee44 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:04.908100 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853bc826-583c-4636-bd91-7c1e0bd1ee44 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:04.908275 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853bc826-583c-4636-bd91-7c1e0bd1ee44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:04.926546 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-853bc826-583c-4636-bd91-7c1e0bd1ee44 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:10:04.927770 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-853bc826-583c-4636-bd91-7c1e0bd1ee44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:10:04.956484 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:10:04.994888 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 584/1171] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:10:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 09:10:05.278586 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923f766d-c8c4-4725-88ad-14592741063d None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:10:05.376051 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:10:05.379811 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923f766d-c8c4-4725-88ad-14592741063d None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:10:05.411943 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1680d2c5-6e2b-4828-8579-79c3c3861dd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:10:05.412566 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1680d2c5-6e2b-4828-8579-79c3c3861dd5 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:05.412844 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1680d2c5-6e2b-4828-8579-79c3c3861dd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:05.413116 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1680d2c5-6e2b-4828-8579-79c3c3861dd5 None admin] PATH_INFO: `/v3/domains/f069831d44dd44f3b1f6d94f3f585c2a` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:05.423816 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1680d2c5-6e2b-4828-8579-79c3c3861dd5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=f069831d44dd44f3b1f6d94f3f585c2a)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:10:05.425252 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1680d2c5-6e2b-4828-8579-79c3c3861dd5 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:10:05.426827 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 588/1172] 10.222.0.16 () {56 vars in 1183 bytes} [Wed Aug 24 09:10:05 2022] GET /identity/v3/domains/f069831d44dd44f3b1f6d94f3f585c2a => generated 267 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:10:05.433184 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853bc826-583c-4636-bd91-7c1e0bd1ee44 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:10:05.540740 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853bc826-583c-4636-bd91-7c1e0bd1ee44 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:10:05.574353 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea658785-1de3-4d0c-ba43-2eb86440f340 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:10:05.574954 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea658785-1de3-4d0c-ba43-2eb86440f340 None admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:05.575230 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea658785-1de3-4d0c-ba43-2eb86440f340 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:05.575495 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea658785-1de3-4d0c-ba43-2eb86440f340 None admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:05.576165 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea658785-1de3-4d0c-ba43-2eb86440f340 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:10:05.577583 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea658785-1de3-4d0c-ba43-2eb86440f340 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:10:05.644692 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 585/1173] 10.222.0.16 () {60 vars in 1141 bytes} [Wed Aug 24 09:10:05 2022] POST /identity/v3/projects => generated 361 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:10:07.560309 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a207e45-f27b-45bc-bcc5-2d47ca5b6025 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:07.560816 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a207e45-f27b-45bc-bcc5-2d47ca5b6025 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:07.560816 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a207e45-f27b-45bc-bcc5-2d47ca5b6025 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:07.562087 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 589/1174] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:10:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:10:07.567435 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ba1f4a-1af5-460d-9433-41ecc3a6a292 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:07.567906 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ba1f4a-1af5-460d-9433-41ecc3a6a292 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:07.568092 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ba1f4a-1af5-460d-9433-41ecc3a6a292 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:07.585994 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-13ba1f4a-1af5-460d-9433-41ecc3a6a292 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:10:07.587180 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-13ba1f4a-1af5-460d-9433-41ecc3a6a292 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:10:07.597657 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:10:07.652237 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 586/1175] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:10:07 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 09:10:07.928190 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a207e45-f27b-45bc-bcc5-2d47ca5b6025 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:10:08.026003 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a207e45-f27b-45bc-bcc5-2d47ca5b6025 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:10:08.057947 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d89875d-935a-4712-a058-ad6be5f25185 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:10:08.058554 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d89875d-935a-4712-a058-ad6be5f25185 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:08.058830 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d89875d-935a-4712-a058-ad6be5f25185 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:08.059099 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d89875d-935a-4712-a058-ad6be5f25185 None admin] PATH_INFO: `/v3/domains/f069831d44dd44f3b1f6d94f3f585c2a` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:08.060195 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d89875d-935a-4712-a058-ad6be5f25185 None admin] RBAC: Authorizing `identity:get_domain(domain_id=f069831d44dd44f3b1f6d94f3f585c2a)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:10:08.061608 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d89875d-935a-4712-a058-ad6be5f25185 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:10:08.063128 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 590/1176] 10.222.0.16 () {56 vars in 1183 bytes} [Wed Aug 24 09:10:07 2022] GET /identity/v3/domains/f069831d44dd44f3b1f6d94f3f585c2a => generated 267 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:10:08.068678 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ba1f4a-1af5-460d-9433-41ecc3a6a292 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:10:08.163831 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ba1f4a-1af5-460d-9433-41ecc3a6a292 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:10:08.196522 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83a6c23-e615-4f06-8f51-789e686fc2ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:10:08.197063 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83a6c23-e615-4f06-8f51-789e686fc2ce None admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:08.197339 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83a6c23-e615-4f06-8f51-789e686fc2ce None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:08.197610 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83a6c23-e615-4f06-8f51-789e686fc2ce None admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:08.198251 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83a6c23-e615-4f06-8f51-789e686fc2ce None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:10:08.199686 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83a6c23-e615-4f06-8f51-789e686fc2ce None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:10:08.264127 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 587/1177] 10.222.0.16 () {60 vars in 1132 bytes} [Wed Aug 24 09:10:08 2022] POST /identity/v3/users => generated 311 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:10:10.203484 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8a063d-54e6-4764-be4f-827272062153 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:10.204162 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8a063d-54e6-4764-be4f-827272062153 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:10.204162 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8a063d-54e6-4764-be4f-827272062153 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:10.205407 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 591/1178] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:10:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:10:10.211182 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae12ab8-2fed-454a-9bbe-787b4c6024d5 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:10.211576 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae12ab8-2fed-454a-9bbe-787b4c6024d5 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:10.211755 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae12ab8-2fed-454a-9bbe-787b4c6024d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:10.217981 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:10:10.239208 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-6ae12ab8-2fed-454a-9bbe-787b4c6024d5 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:10:10.243765 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-6ae12ab8-2fed-454a-9bbe-787b4c6024d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:10:10.308281 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 588/1179] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:10:10 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 09:10:10.582228 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8a063d-54e6-4764-be4f-827272062153 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:10:10.610919 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:10:10.682614 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8a063d-54e6-4764-be4f-827272062153 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:10:10.714952 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b94a25-426c-44e8-b0a2-51ebb7ee3024 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:10:10.715572 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b94a25-426c-44e8-b0a2-51ebb7ee3024 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:10.715885 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b94a25-426c-44e8-b0a2-51ebb7ee3024 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:10.716162 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b94a25-426c-44e8-b0a2-51ebb7ee3024 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:10.726979 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b94a25-426c-44e8-b0a2-51ebb7ee3024 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:10:10.731601 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-d9b94a25-426c-44e8-b0a2-51ebb7ee3024 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:10:10.731601 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:10:10.731601 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:10:10.731601 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:10:10.731601 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:10:10.731601 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:10:10.731601 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:10:10.731601 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:10:10.731601 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:10:10.731601 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:10:10.731601 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:10:10.731601 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:10:10.731601 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:10:10.731601 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:10:10.731601 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:10:10.731601 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:10:10.735721 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:10:10.735721 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:10:10.735721 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:10:10.735721 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:10:10.735721 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:10:10.735721 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:10:10.735721 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:10:10.735721 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:10:10.735721 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:10:10.735721 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:10:10.735721 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:10:10.735721 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Aug 24 09:10:10.735721 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:10:10.735721 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:10:10.735721 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:10:10.735721 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:10:10.736890 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:10:10.736890 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:10:10.736890 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:10:10.736890 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:10:10.736890 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:10:10.736890 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:10:10.736890 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:10:10.736890 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:10:10.736890 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:10:10.736890 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:10:10.736890 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:10:10.736890 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Aug 24 09:10:10.736890 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 592/1180] 10.222.0.16 () {56 vars in 1096 bytes} [Wed Aug 24 09:10:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:10:10.738656 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae12ab8-2fed-454a-9bbe-787b4c6024d5 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:10:10.837222 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae12ab8-2fed-454a-9bbe-787b4c6024d5 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:10:10.865997 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:10:10.870746 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499a15f5-5313-4745-9465-76f059ca713d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:10:10.871352 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499a15f5-5313-4745-9465-76f059ca713d None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:10.871656 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499a15f5-5313-4745-9465-76f059ca713d None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:10.871930 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499a15f5-5313-4745-9465-76f059ca713d None admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:10.872591 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499a15f5-5313-4745-9465-76f059ca713d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:10:10.872893 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499a15f5-5313-4745-9465-76f059ca713d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:10:10.874251 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499a15f5-5313-4745-9465-76f059ca713d None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:10:10.882069 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 589/1181] 10.222.0.16 () {56 vars in 1099 bytes} [Wed Aug 24 09:10:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:10:10.888278 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b94a25-426c-44e8-b0a2-51ebb7ee3024 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:10:10.974586 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:10:10.986173 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b94a25-426c-44e8-b0a2-51ebb7ee3024 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:10:11.019155 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b68e58-5c26-439c-95f6-5d90f590a60a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:10:11.019795 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b68e58-5c26-439c-95f6-5d90f590a60a None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:11.020074 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b68e58-5c26-439c-95f6-5d90f590a60a None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:11.020344 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b68e58-5c26-439c-95f6-5d90f590a60a None admin] PATH_INFO: `/v3/users/63c351a5fc2c48ea8cc9ab1bbe9ca1f0` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:11.032885 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b68e58-5c26-439c-95f6-5d90f590a60a None admin] RBAC: Authorizing `identity:get_user(user_id=63c351a5fc2c48ea8cc9ab1bbe9ca1f0)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:10:11.034344 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b68e58-5c26-439c-95f6-5d90f590a60a None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:10:11.035939 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 593/1182] 10.222.0.16 () {56 vars in 1177 bytes} [Wed Aug 24 09:10:10 2022] GET /identity/v3/users/63c351a5fc2c48ea8cc9ab1bbe9ca1f0 => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:10:11.041844 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499a15f5-5313-4745-9465-76f059ca713d None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:10:11.139354 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499a15f5-5313-4745-9465-76f059ca713d None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:10:11.180207 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98eb5b2c-6c7f-4402-83ea-29a246ce23f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:10:11.180810 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98eb5b2c-6c7f-4402-83ea-29a246ce23f1 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:11.181088 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98eb5b2c-6c7f-4402-83ea-29a246ce23f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:11.181359 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98eb5b2c-6c7f-4402-83ea-29a246ce23f1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:11.302878 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:10:11.310701 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98eb5b2c-6c7f-4402-83ea-29a246ce23f1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:10:11.312179 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98eb5b2c-6c7f-4402-83ea-29a246ce23f1 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:10:11.407424 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 590/1183] 10.222.0.16 () {58 vars in 1282 bytes} [Wed Aug 24 09:10:11 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 09:10:11.413810 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b68e58-5c26-439c-95f6-5d90f590a60a None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:10:11.510704 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b68e58-5c26-439c-95f6-5d90f590a60a None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:10:11.544936 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6a3502-5210-4430-b64e-65d5873d5715 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:10:11.545563 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6a3502-5210-4430-b64e-65d5873d5715 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:11.545845 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6a3502-5210-4430-b64e-65d5873d5715 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:11.546121 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6a3502-5210-4430-b64e-65d5873d5715 None admin] PATH_INFO: `/v3/projects/447947acd2c441f3a95ab10cc3b810d5` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:11.547233 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6a3502-5210-4430-b64e-65d5873d5715 None admin] RBAC: Authorizing `identity:get_project(project_id=447947acd2c441f3a95ab10cc3b810d5)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:10:11.548696 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6a3502-5210-4430-b64e-65d5873d5715 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:10:11.550316 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 594/1184] 10.222.0.16 () {56 vars in 1186 bytes} [Wed Aug 24 09:10:11 2022] GET /identity/v3/projects/447947acd2c441f3a95ab10cc3b810d5 => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:10:11.556448 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98eb5b2c-6c7f-4402-83ea-29a246ce23f1 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:10:11.646468 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:10:11.652948 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98eb5b2c-6c7f-4402-83ea-29a246ce23f1 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:10:11.686031 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f2a3e9-9d76-4da5-b307-12e9dd1d073d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:10:11.686561 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f2a3e9-9d76-4da5-b307-12e9dd1d073d None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:11.686842 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f2a3e9-9d76-4da5-b307-12e9dd1d073d None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:11.687112 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f2a3e9-9d76-4da5-b307-12e9dd1d073d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:11.687870 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f2a3e9-9d76-4da5-b307-12e9dd1d073d None admin] RBAC: Adding query filter params (role.id=15f634d4a2d247ddab90c458e5e1c141, scope.project.id=447947acd2c441f3a95ab10cc3b810d5, user.id=63c351a5fc2c48ea8cc9ab1bbe9ca1f0) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:10:11.688177 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f2a3e9-9d76-4da5-b307-12e9dd1d073d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:10:11.689591 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f2a3e9-9d76-4da5-b307-12e9dd1d073d None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:10:11.695353 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 591/1185] 10.222.0.16 () {56 vars in 1374 bytes} [Wed Aug 24 09:10:11 2022] GET /identity/v3/role_assignments?user.id=63c351a5fc2c48ea8cc9ab1bbe9ca1f0&scope.project.id=447947acd2c441f3a95ab10cc3b810d5&role.id=15f634d4a2d247ddab90c458e5e1c141 => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:10:13.564759 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3246b57-1d0b-4196-abd8-8260216911f7 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:13.565277 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3246b57-1d0b-4196-abd8-8260216911f7 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:13.565277 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3246b57-1d0b-4196-abd8-8260216911f7 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:13.566546 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 595/1186] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:10:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:10:13.572216 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688d911c-bba7-4078-8d78-cda6c6da7447 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:13.572635 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688d911c-bba7-4078-8d78-cda6c6da7447 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:13.572767 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688d911c-bba7-4078-8d78-cda6c6da7447 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:13.591325 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-688d911c-bba7-4078-8d78-cda6c6da7447 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:10:13.592542 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-688d911c-bba7-4078-8d78-cda6c6da7447 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:10:13.657095 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 592/1187] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:10:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 09:10:13.931867 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3246b57-1d0b-4196-abd8-8260216911f7 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:10:13.947930 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:10:14.029311 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3246b57-1d0b-4196-abd8-8260216911f7 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:10:14.061540 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d5e937-3fb8-403a-80e8-d947e2d6e380 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:10:14.062153 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d5e937-3fb8-403a-80e8-d947e2d6e380 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:14.062429 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d5e937-3fb8-403a-80e8-d947e2d6e380 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:14.062697 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d5e937-3fb8-403a-80e8-d947e2d6e380 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:14.073016 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d5e937-3fb8-403a-80e8-d947e2d6e380 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:10:14.077833 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-45d5e937-3fb8-403a-80e8-d947e2d6e380 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:10:14.077833 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:10:14.077833 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:10:14.077833 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:10:14.077833 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:10:14.077833 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:10:14.077833 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:10:14.077833 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:10:14.077833 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:10:14.077833 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:10:14.077833 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:10:14.077833 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:10:14.077833 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:10:14.077833 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:10:14.077833 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:10:14.077833 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:10:14.078940 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:10:14.078940 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:10:14.078940 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:10:14.078940 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:10:14.078940 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:10:14.078940 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:10:14.078940 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:10:14.078940 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:10:14.078940 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:10:14.078940 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:10:14.078940 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:10:14.078940 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Aug 24 09:10:14.078940 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:10:14.078940 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:10:14.078940 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:10:14.078940 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:10:14.080195 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:10:14.080195 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:10:14.080195 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:10:14.080195 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:10:14.080195 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:10:14.080195 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:10:14.080195 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:10:14.080195 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:10:14.080195 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:10:14.080195 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:10:14.080195 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:10:14.080195 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Aug 24 09:10:14.080195 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 596/1188] 10.222.0.16 () {56 vars in 1096 bytes} [Wed Aug 24 09:10:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:10:14.084821 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688d911c-bba7-4078-8d78-cda6c6da7447 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:10:14.168782 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:10:14.180983 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688d911c-bba7-4078-8d78-cda6c6da7447 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:10:14.213934 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2acf86f-2a5a-491b-ba71-988ede947c4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:10:14.214490 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2acf86f-2a5a-491b-ba71-988ede947c4d None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:14.214765 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2acf86f-2a5a-491b-ba71-988ede947c4d None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:14.215036 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2acf86f-2a5a-491b-ba71-988ede947c4d None admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:14.215718 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2acf86f-2a5a-491b-ba71-988ede947c4d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:10:14.216021 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2acf86f-2a5a-491b-ba71-988ede947c4d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:10:14.217366 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2acf86f-2a5a-491b-ba71-988ede947c4d None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:10:14.225180 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 593/1189] 10.222.0.16 () {56 vars in 1099 bytes} [Wed Aug 24 09:10:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:10:14.231450 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d5e937-3fb8-403a-80e8-d947e2d6e380 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:10:14.305580 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:10:14.329789 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d5e937-3fb8-403a-80e8-d947e2d6e380 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:10:14.363267 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337eca77-c69a-402b-b45e-94c7bdc63c41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:10:14.363911 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337eca77-c69a-402b-b45e-94c7bdc63c41 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:14.364192 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337eca77-c69a-402b-b45e-94c7bdc63c41 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:14.364465 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337eca77-c69a-402b-b45e-94c7bdc63c41 None admin] PATH_INFO: `/v3/users/63c351a5fc2c48ea8cc9ab1bbe9ca1f0` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:14.365601 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337eca77-c69a-402b-b45e-94c7bdc63c41 None admin] RBAC: Authorizing `identity:get_user(user_id=63c351a5fc2c48ea8cc9ab1bbe9ca1f0)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:10:14.367684 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337eca77-c69a-402b-b45e-94c7bdc63c41 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:10:14.369244 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 597/1190] 10.222.0.16 () {56 vars in 1177 bytes} [Wed Aug 24 09:10:14 2022] GET /identity/v3/users/63c351a5fc2c48ea8cc9ab1bbe9ca1f0 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:10:14.375057 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2acf86f-2a5a-491b-ba71-988ede947c4d None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:10:14.472129 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2acf86f-2a5a-491b-ba71-988ede947c4d None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:10:14.504480 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16181749-98f2-4155-ae35-3ff06e2be425 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:10:14.505094 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16181749-98f2-4155-ae35-3ff06e2be425 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:14.505378 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16181749-98f2-4155-ae35-3ff06e2be425 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:14.505654 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16181749-98f2-4155-ae35-3ff06e2be425 None admin] PATH_INFO: `/v3/projects/447947acd2c441f3a95ab10cc3b810d5` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:14.506766 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16181749-98f2-4155-ae35-3ff06e2be425 None admin] RBAC: Authorizing `identity:get_project(project_id=447947acd2c441f3a95ab10cc3b810d5)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:10:14.508254 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16181749-98f2-4155-ae35-3ff06e2be425 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:10:14.509866 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 594/1191] 10.222.0.16 () {56 vars in 1186 bytes} [Wed Aug 24 09:10:14 2022] GET /identity/v3/projects/447947acd2c441f3a95ab10cc3b810d5 => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:10:14.515666 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337eca77-c69a-402b-b45e-94c7bdc63c41 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:10:14.614712 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337eca77-c69a-402b-b45e-94c7bdc63c41 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:10:14.647093 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d23acf3-ec14-4069-bd8f-fe2011d8f327 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:10:14.647763 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d23acf3-ec14-4069-bd8f-fe2011d8f327 None admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:14.648558 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d23acf3-ec14-4069-bd8f-fe2011d8f327 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:14.648832 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d23acf3-ec14-4069-bd8f-fe2011d8f327 None admin] PATH_INFO: `/v3/projects/447947acd2c441f3a95ab10cc3b810d5/users/63c351a5fc2c48ea8cc9ab1bbe9ca1f0/roles/15f634d4a2d247ddab90c458e5e1c141` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:14.650696 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d23acf3-ec14-4069-bd8f-fe2011d8f327 None admin] RBAC: Authorizing `identity:create_grant(project_id=447947acd2c441f3a95ab10cc3b810d5, user_id=63c351a5fc2c48ea8cc9ab1bbe9ca1f0, role_id=15f634d4a2d247ddab90c458e5e1c141)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:10:14.652145 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d23acf3-ec14-4069-bd8f-fe2011d8f327 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:10:14.717553 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 598/1192] 10.222.0.16 () {58 vars in 1439 bytes} [Wed Aug 24 09:10:14 2022] PUT /identity/v3/projects/447947acd2c441f3a95ab10cc3b810d5/users/63c351a5fc2c48ea8cc9ab1bbe9ca1f0/roles/15f634d4a2d247ddab90c458e5e1c141 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:10:16.581604 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8545b714-e961-49a3-99d9-8854113c64c4 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:16.582119 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8545b714-e961-49a3-99d9-8854113c64c4 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:16.582119 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8545b714-e961-49a3-99d9-8854113c64c4 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:16.583405 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 595/1193] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:10:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:10:16.589434 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35d1ac4-e5e4-462d-b3d2-2c1643786812 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:16.589852 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35d1ac4-e5e4-462d-b3d2-2c1643786812 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:16.589990 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35d1ac4-e5e4-462d-b3d2-2c1643786812 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:16.596273 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:10:16.608700 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-e35d1ac4-e5e4-462d-b3d2-2c1643786812 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:10:16.614032 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-e35d1ac4-e5e4-462d-b3d2-2c1643786812 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:10:16.674659 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 599/1194] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:10:16 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 09:10:16.944247 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8545b714-e961-49a3-99d9-8854113c64c4 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:10:16.964114 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:10:17.044608 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8545b714-e961-49a3-99d9-8854113c64c4 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:10:17.078099 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c718b6-8f58-4c93-8f70-12d56b061e7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:10:17.078716 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c718b6-8f58-4c93-8f70-12d56b061e7c None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:17.079026 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c718b6-8f58-4c93-8f70-12d56b061e7c None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:17.079303 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c718b6-8f58-4c93-8f70-12d56b061e7c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:17.089699 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c718b6-8f58-4c93-8f70-12d56b061e7c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:10:17.094350 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-e2c718b6-8f58-4c93-8f70-12d56b061e7c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:10:17.094350 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:10:17.094350 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:10:17.094350 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:10:17.094350 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:10:17.094350 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:10:17.094350 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:10:17.094350 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:10:17.094350 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:10:17.094350 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:10:17.094350 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:10:17.094350 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:10:17.094350 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:10:17.094350 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:10:17.094350 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:10:17.094350 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:10:17.095446 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:10:17.095446 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:10:17.095446 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:10:17.095446 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:10:17.095446 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:10:17.095446 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:10:17.095446 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:10:17.095446 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:10:17.095446 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:10:17.095446 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:10:17.095446 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:10:17.095446 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 24 09:10:17.095446 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:10:17.095446 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:10:17.095446 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:10:17.095446 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:10:17.096541 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:10:17.096541 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:10:17.096541 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:10:17.096541 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:10:17.096541 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:10:17.096541 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:10:17.096541 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:10:17.096541 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:10:17.096541 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:10:17.096541 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:10:17.096541 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:10:17.096541 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 24 09:10:17.096541 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 596/1195] 10.222.0.16 () {56 vars in 1096 bytes} [Wed Aug 24 09:10:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:10:17.101489 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35d1ac4-e5e4-462d-b3d2-2c1643786812 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:10:17.197677 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35d1ac4-e5e4-462d-b3d2-2c1643786812 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:10:17.225893 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:10:17.231146 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d823b6c2-2429-4c73-81ef-f7d9a5a933e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:10:17.231779 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d823b6c2-2429-4c73-81ef-f7d9a5a933e1 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:17.232064 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d823b6c2-2429-4c73-81ef-f7d9a5a933e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:17.232340 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d823b6c2-2429-4c73-81ef-f7d9a5a933e1 None admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:17.233046 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d823b6c2-2429-4c73-81ef-f7d9a5a933e1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:10:17.233354 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d823b6c2-2429-4c73-81ef-f7d9a5a933e1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:10:17.234712 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d823b6c2-2429-4c73-81ef-f7d9a5a933e1 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:10:17.243159 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 600/1196] 10.222.0.16 () {56 vars in 1099 bytes} [Wed Aug 24 09:10:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:10:17.248899 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c718b6-8f58-4c93-8f70-12d56b061e7c None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:10:17.325573 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:10:17.346475 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c718b6-8f58-4c93-8f70-12d56b061e7c None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:10:17.380165 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058b7a3d-44d1-4dff-af06-03afcc6671e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:10:17.380786 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058b7a3d-44d1-4dff-af06-03afcc6671e6 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:17.381067 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058b7a3d-44d1-4dff-af06-03afcc6671e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:17.381339 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058b7a3d-44d1-4dff-af06-03afcc6671e6 None admin] PATH_INFO: `/v3/users/63c351a5fc2c48ea8cc9ab1bbe9ca1f0` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:17.382454 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058b7a3d-44d1-4dff-af06-03afcc6671e6 None admin] RBAC: Authorizing `identity:get_user(user_id=63c351a5fc2c48ea8cc9ab1bbe9ca1f0)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:10:17.383938 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058b7a3d-44d1-4dff-af06-03afcc6671e6 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:10:17.385499 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 597/1197] 10.222.0.16 () {56 vars in 1177 bytes} [Wed Aug 24 09:10:17 2022] GET /identity/v3/users/63c351a5fc2c48ea8cc9ab1bbe9ca1f0 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:10:17.391354 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d823b6c2-2429-4c73-81ef-f7d9a5a933e1 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:10:17.489092 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d823b6c2-2429-4c73-81ef-f7d9a5a933e1 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:10:17.520458 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64a9da2-6787-411e-977f-57a8748d8f2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:10:17.521041 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64a9da2-6787-411e-977f-57a8748d8f2b None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:17.521317 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64a9da2-6787-411e-977f-57a8748d8f2b None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:17.521585 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64a9da2-6787-411e-977f-57a8748d8f2b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:17.579230 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:10:17.586695 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64a9da2-6787-411e-977f-57a8748d8f2b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:10:17.588198 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64a9da2-6787-411e-977f-57a8748d8f2b None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:10:17.707681 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 601/1198] 10.222.0.16 () {58 vars in 1282 bytes} [Wed Aug 24 09:10:17 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 09:10:17.714123 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058b7a3d-44d1-4dff-af06-03afcc6671e6 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:10:17.809154 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058b7a3d-44d1-4dff-af06-03afcc6671e6 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:10:17.842244 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722f9ded-6f74-4b20-87f2-57b28d16045c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:10:17.842855 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722f9ded-6f74-4b20-87f2-57b28d16045c None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:17.843129 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722f9ded-6f74-4b20-87f2-57b28d16045c None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:17.843397 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722f9ded-6f74-4b20-87f2-57b28d16045c None admin] PATH_INFO: `/v3/projects/447947acd2c441f3a95ab10cc3b810d5` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:17.844522 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722f9ded-6f74-4b20-87f2-57b28d16045c None admin] RBAC: Authorizing `identity:get_project(project_id=447947acd2c441f3a95ab10cc3b810d5)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:10:17.845927 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722f9ded-6f74-4b20-87f2-57b28d16045c None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:10:17.847507 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 598/1199] 10.222.0.16 () {56 vars in 1186 bytes} [Wed Aug 24 09:10:17 2022] GET /identity/v3/projects/447947acd2c441f3a95ab10cc3b810d5 => generated 361 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:10:17.853508 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64a9da2-6787-411e-977f-57a8748d8f2b None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:10:17.944409 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:10:17.950692 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64a9da2-6787-411e-977f-57a8748d8f2b None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:10:17.985957 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4f5a8b-dd85-41b6-9db4-57a6e20c7217 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:10:17.986557 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4f5a8b-dd85-41b6-9db4-57a6e20c7217 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:10:17.986835 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4f5a8b-dd85-41b6-9db4-57a6e20c7217 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:10:17.987104 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4f5a8b-dd85-41b6-9db4-57a6e20c7217 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:10:17.987852 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4f5a8b-dd85-41b6-9db4-57a6e20c7217 None admin] RBAC: Adding query filter params (role.id=15f634d4a2d247ddab90c458e5e1c141, scope.project.id=447947acd2c441f3a95ab10cc3b810d5, user.id=63c351a5fc2c48ea8cc9ab1bbe9ca1f0) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:10:17.988158 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4f5a8b-dd85-41b6-9db4-57a6e20c7217 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:10:17.989584 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4f5a8b-dd85-41b6-9db4-57a6e20c7217 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:10:17.995769 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 602/1200] 10.222.0.16 () {56 vars in 1374 bytes} [Wed Aug 24 09:10:17 2022] GET /identity/v3/role_assignments?user.id=63c351a5fc2c48ea8cc9ab1bbe9ca1f0&scope.project.id=447947acd2c441f3a95ab10cc3b810d5&role.id=15f634d4a2d247ddab90c458e5e1c141 => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:10:27.838227 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=93 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:10:27.992635 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=21 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:11:32.808346 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45637720-8213-4bcc-a94d-906dd7520b5d None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:11:32.808934 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45637720-8213-4bcc-a94d-906dd7520b5d None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:11:32.808934 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45637720-8213-4bcc-a94d-906dd7520b5d None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:11:32.810235 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 599/1201] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:11:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:11:32.815568 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7a5ff3-f00f-426b-9f57-486f6eca6af8 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:11:32.816022 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7a5ff3-f00f-426b-9f57-486f6eca6af8 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:11:32.816159 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7a5ff3-f00f-426b-9f57-486f6eca6af8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:11:32.834159 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-5e7a5ff3-f00f-426b-9f57-486f6eca6af8 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:11:32.835357 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-5e7a5ff3-f00f-426b-9f57-486f6eca6af8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:11:32.902245 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 603/1202] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:11:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 09:11:33.171666 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45637720-8213-4bcc-a94d-906dd7520b5d None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:11:33.272539 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45637720-8213-4bcc-a94d-906dd7520b5d None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:11:33.306294 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c877d9d3-0979-4325-a4fb-e1f156df9dd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:11:33.306929 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c877d9d3-0979-4325-a4fb-e1f156df9dd3 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:11:33.307204 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c877d9d3-0979-4325-a4fb-e1f156df9dd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:11:33.307470 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c877d9d3-0979-4325-a4fb-e1f156df9dd3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:11:33.317949 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c877d9d3-0979-4325-a4fb-e1f156df9dd3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:11:33.319376 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c877d9d3-0979-4325-a4fb-e1f156df9dd3 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:11:33.334334 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-c877d9d3-0979-4325-a4fb-e1f156df9dd3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:11:33.334334 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:11:33.334334 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:11:33.334334 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:11:33.334334 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:11:33.334334 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:11:33.334334 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:11:33.334334 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:11:33.334334 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:11:33.334334 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:11:33.334334 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:11:33.334334 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:11:33.334334 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:11:33.334334 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:11:33.334334 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:11:33.334334 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:11:33.335538 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:11:33.335538 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:11:33.335538 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:11:33.335538 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:11:33.335538 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:11:33.335538 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:11:33.335538 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:11:33.335538 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 24 09:11:33.335538 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:11:33.335538 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:11:33.335538 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:11:33.335538 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:11:33.335538 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:11:33.335538 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:11:33.335538 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:11:33.335538 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:11:33.335538 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:11:33.336587 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:11:33.336587 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:11:33.336587 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 24 09:11:33.336587 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 600/1203] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 24 09:11:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:11:33.341364 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7a5ff3-f00f-426b-9f57-486f6eca6af8 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:11:33.437144 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7a5ff3-f00f-426b-9f57-486f6eca6af8 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:11:33.465833 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:11:33.471601 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac58d669-a7a4-42f7-8b19-2dadc9072f01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:11:33.472237 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac58d669-a7a4-42f7-8b19-2dadc9072f01 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:11:33.472503 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac58d669-a7a4-42f7-8b19-2dadc9072f01 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:11:33.472777 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac58d669-a7a4-42f7-8b19-2dadc9072f01 None admin] PATH_INFO: `/v3/domains` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:11:33.473409 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac58d669-a7a4-42f7-8b19-2dadc9072f01 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:11:33.473705 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac58d669-a7a4-42f7-8b19-2dadc9072f01 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:11:33.475051 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac58d669-a7a4-42f7-8b19-2dadc9072f01 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:11:33.485849 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 604/1204] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Aug 24 09:11:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:11:33.491616 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c877d9d3-0979-4325-a4fb-e1f156df9dd3 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:11:33.544050 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:11:33.589345 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c877d9d3-0979-4325-a4fb-e1f156df9dd3 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:11:33.622950 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917a125c-872e-4ca9-97ff-b88ef0341c3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:11:33.623551 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917a125c-872e-4ca9-97ff-b88ef0341c3f None admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:11:33.623858 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917a125c-872e-4ca9-97ff-b88ef0341c3f None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:11:33.624130 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917a125c-872e-4ca9-97ff-b88ef0341c3f None admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:11:33.624765 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917a125c-872e-4ca9-97ff-b88ef0341c3f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:11:33.626204 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917a125c-872e-4ca9-97ff-b88ef0341c3f None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:11:33.722220 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 601/1205] 10.222.0.16 () {60 vars in 1132 bytes} [Wed Aug 24 09:11:33 2022] POST /identity/v3/users => generated 251 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:11:35.620378 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fee07e0-9123-49c3-ba13-22d576209955 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:11:35.620902 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fee07e0-9123-49c3-ba13-22d576209955 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:11:35.620902 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fee07e0-9123-49c3-ba13-22d576209955 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:11:35.622164 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 605/1206] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:11:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:11:35.627302 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d614978b-113d-4aff-b629-6f76feebb2a8 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:11:35.627774 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d614978b-113d-4aff-b629-6f76feebb2a8 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:11:35.627902 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d614978b-113d-4aff-b629-6f76feebb2a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:11:35.645859 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-d614978b-113d-4aff-b629-6f76feebb2a8 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:11:35.647328 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-d614978b-113d-4aff-b629-6f76feebb2a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:11:35.714061 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 602/1207] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:11:35 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 09:11:35.980977 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fee07e0-9123-49c3-ba13-22d576209955 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:11:36.082271 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fee07e0-9123-49c3-ba13-22d576209955 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:11:36.114987 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c15cf68-00fc-46f2-b1d9-57c1012e9654 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:11:36.115626 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c15cf68-00fc-46f2-b1d9-57c1012e9654 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:11:36.115909 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c15cf68-00fc-46f2-b1d9-57c1012e9654 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:11:36.116183 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c15cf68-00fc-46f2-b1d9-57c1012e9654 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:11:36.126547 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c15cf68-00fc-46f2-b1d9-57c1012e9654 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:11:36.131180 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-1c15cf68-00fc-46f2-b1d9-57c1012e9654 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 09:11:36.131180 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:11:36.131180 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:11:36.131180 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:11:36.131180 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:11:36.131180 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:11:36.131180 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:11:36.131180 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:11:36.131180 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:11:36.131180 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:11:36.131180 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:11:36.131180 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:11:36.131180 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:11:36.131180 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:11:36.131180 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:11:36.131180 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:11:36.132972 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:11:36.132972 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:11:36.132972 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:11:36.132972 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:11:36.132972 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:11:36.132972 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:11:36.132972 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:11:36.132972 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:11:36.132972 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:11:36.132972 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:11:36.132972 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:11:36.132972 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Aug 24 09:11:36.132972 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:11:36.132972 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:11:36.132972 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:11:36.132972 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:11:36.134153 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:11:36.134153 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:11:36.134153 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:11:36.134153 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:11:36.134153 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:11:36.134153 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:11:36.134153 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:11:36.134153 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:11:36.134153 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:11:36.134153 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:11:36.134153 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 09:11:36.134153 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Aug 24 09:11:36.134153 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 606/1208] 10.222.0.16 () {56 vars in 1102 bytes} [Wed Aug 24 09:11:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:11:36.138369 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d614978b-113d-4aff-b629-6f76feebb2a8 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:11:36.177314 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:11:36.233754 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d614978b-113d-4aff-b629-6f76feebb2a8 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:11:36.266187 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef7af17-058f-4f8a-a475-97a7d616a136 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:11:36.271678 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef7af17-058f-4f8a-a475-97a7d616a136 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:11:36.271678 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef7af17-058f-4f8a-a475-97a7d616a136 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:11:36.271678 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef7af17-058f-4f8a-a475-97a7d616a136 None admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:11:36.271678 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef7af17-058f-4f8a-a475-97a7d616a136 None admin] RBAC: Adding query filter params (name=service) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:11:36.271678 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef7af17-058f-4f8a-a475-97a7d616a136 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:11:36.271678 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef7af17-058f-4f8a-a475-97a7d616a136 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:11:36.277362 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 603/1209] 10.222.0.16 () {56 vars in 1103 bytes} [Wed Aug 24 09:11:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:11:36.283845 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c15cf68-00fc-46f2-b1d9-57c1012e9654 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:11:36.334295 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:11:36.382805 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c15cf68-00fc-46f2-b1d9-57c1012e9654 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:11:36.416125 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a77bbba-22dc-4111-a78d-c17e0d080d13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:11:36.416761 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a77bbba-22dc-4111-a78d-c17e0d080d13 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:11:36.417059 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a77bbba-22dc-4111-a78d-c17e0d080d13 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:11:36.417327 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a77bbba-22dc-4111-a78d-c17e0d080d13 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:11:36.427785 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a77bbba-22dc-4111-a78d-c17e0d080d13 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:11:36.429318 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a77bbba-22dc-4111-a78d-c17e0d080d13 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:11:36.443543 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-8a77bbba-22dc-4111-a78d-c17e0d080d13 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:11:36.443543 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:11:36.443543 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:11:36.443543 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:11:36.443543 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:11:36.443543 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:11:36.443543 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:11:36.443543 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:11:36.443543 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:11:36.443543 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:11:36.443543 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:11:36.443543 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:11:36.443543 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:11:36.443543 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:11:36.443543 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:11:36.443543 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:11:36.444700 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:11:36.444700 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:11:36.444700 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:11:36.444700 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:11:36.444700 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:11:36.444700 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:11:36.444700 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:11:36.444700 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Aug 24 09:11:36.444700 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:11:36.444700 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:11:36.444700 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:11:36.444700 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:11:36.444700 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:11:36.444700 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:11:36.444700 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:11:36.444700 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:11:36.444700 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:11:36.445829 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:11:36.445829 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:11:36.445829 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Aug 24 09:11:36.445829 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 607/1210] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 24 09:11:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:11:36.450761 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef7af17-058f-4f8a-a475-97a7d616a136 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:11:36.548381 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:11:36.551422 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef7af17-058f-4f8a-a475-97a7d616a136 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:11:36.585623 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8050622-03c3-42bd-852a-7aed8772746f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:11:36.586230 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8050622-03c3-42bd-852a-7aed8772746f None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:11:36.586518 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8050622-03c3-42bd-852a-7aed8772746f None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:11:36.586791 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8050622-03c3-42bd-852a-7aed8772746f None admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:11:36.587432 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8050622-03c3-42bd-852a-7aed8772746f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:11:36.587767 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8050622-03c3-42bd-852a-7aed8772746f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:11:36.589128 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8050622-03c3-42bd-852a-7aed8772746f None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:11:36.600197 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 604/1211] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Aug 24 09:11:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:11:36.606256 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a77bbba-22dc-4111-a78d-c17e0d080d13 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:11:36.706483 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a77bbba-22dc-4111-a78d-c17e0d080d13 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:11:36.708127 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:11:36.740177 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754ef64c-e2cd-4ca0-9f0f-3f93cee032a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:11:36.740799 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754ef64c-e2cd-4ca0-9f0f-3f93cee032a6 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:11:36.741080 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754ef64c-e2cd-4ca0-9f0f-3f93cee032a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:11:36.741355 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754ef64c-e2cd-4ca0-9f0f-3f93cee032a6 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:11:36.748749 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754ef64c-e2cd-4ca0-9f0f-3f93cee032a6 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:11:36.750753 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754ef64c-e2cd-4ca0-9f0f-3f93cee032a6 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:11:36.762163 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-754ef64c-e2cd-4ca0-9f0f-3f93cee032a6 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 09:11:36.762163 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:11:36.762163 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:11:36.762163 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:11:36.762163 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:11:36.762163 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:11:36.762163 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:11:36.762163 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:11:36.762163 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:11:36.762163 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:11:36.762163 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:11:36.762163 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:11:36.762163 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 09:11:36.762163 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 09:11:36.762163 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 09:11:36.762163 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 09:11:36.770962 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:11:36.770962 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:11:36.770962 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 09:11:36.770962 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:11:36.770962 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 09:11:36.770962 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:11:36.770962 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:11:36.770962 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:11:36.770962 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:11:36.770962 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:11:36.770962 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:11:36.770962 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Aug 24 09:11:36.770962 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:11:36.770962 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:11:36.770962 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:11:36.770962 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:11:36.770962 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:11:36.770962 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:11:36.772213 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:11:36.772213 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:11:36.772213 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 09:11:36.772213 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 09:11:36.772213 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 09:11:36.772213 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 09:11:36.772213 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 09:11:36.772213 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 09:11:36.772213 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 09:11:36.772213 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Aug 24 09:11:36.772213 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 608/1212] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 24 09:11:36 2022] GET /identity/v3/users/placement => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:11:36.773044 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8050622-03c3-42bd-852a-7aed8772746f None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:11:36.871289 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8050622-03c3-42bd-852a-7aed8772746f None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:11:36.905975 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfde8aa5-f516-45dd-aa9e-ff1591b35168 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:11:36.906569 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfde8aa5-f516-45dd-aa9e-ff1591b35168 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:11:36.906844 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfde8aa5-f516-45dd-aa9e-ff1591b35168 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:11:36.907112 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfde8aa5-f516-45dd-aa9e-ff1591b35168 None admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:11:36.907858 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfde8aa5-f516-45dd-aa9e-ff1591b35168 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:11:36.908160 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfde8aa5-f516-45dd-aa9e-ff1591b35168 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:11:36.909573 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfde8aa5-f516-45dd-aa9e-ff1591b35168 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:11:36.926768 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 605/1213] 10.222.0.16 () {56 vars in 1143 bytes} [Wed Aug 24 09:11:36 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:11:36.933359 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754ef64c-e2cd-4ca0-9f0f-3f93cee032a6 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:11:37.033914 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754ef64c-e2cd-4ca0-9f0f-3f93cee032a6 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:11:37.066239 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c57e09-2a88-4697-af74-9bda91a9ab48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:11:37.066831 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c57e09-2a88-4697-af74-9bda91a9ab48 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:11:37.067107 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c57e09-2a88-4697-af74-9bda91a9ab48 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:11:37.067374 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c57e09-2a88-4697-af74-9bda91a9ab48 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:11:37.097050 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:11:37.133033 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c57e09-2a88-4697-af74-9bda91a9ab48 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:11:37.134399 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c57e09-2a88-4697-af74-9bda91a9ab48 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:11:37.231788 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 609/1214] 10.222.0.16 () {58 vars in 1282 bytes} [Wed Aug 24 09:11:36 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 09:11:37.237692 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfde8aa5-f516-45dd-aa9e-ff1591b35168 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:11:37.257327 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:11:37.340455 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfde8aa5-f516-45dd-aa9e-ff1591b35168 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:11:37.374031 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f700a1-1054-4152-a090-b989847bc577 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:11:37.374665 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f700a1-1054-4152-a090-b989847bc577 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:11:37.375460 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f700a1-1054-4152-a090-b989847bc577 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:11:37.375772 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f700a1-1054-4152-a090-b989847bc577 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:11:37.385883 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f700a1-1054-4152-a090-b989847bc577 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:11:37.387247 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f700a1-1054-4152-a090-b989847bc577 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:11:37.401837 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-74f700a1-1054-4152-a090-b989847bc577 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:11:37.401837 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:11:37.401837 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:11:37.401837 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:11:37.401837 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:11:37.401837 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:11:37.401837 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:11:37.401837 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:11:37.401837 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:11:37.401837 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:11:37.401837 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:11:37.401837 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:11:37.401837 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:11:37.401837 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:11:37.401837 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:11:37.401837 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:11:37.405699 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:11:37.405699 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:11:37.405699 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:11:37.405699 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:11:37.405699 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:11:37.405699 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:11:37.405699 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:11:37.405699 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 24 09:11:37.405699 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:11:37.405699 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:11:37.405699 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:11:37.405699 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:11:37.405699 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:11:37.405699 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:11:37.405699 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:11:37.405699 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:11:37.405699 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:11:37.406920 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:11:37.406920 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:11:37.406920 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 24 09:11:37.406920 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 606/1215] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 24 09:11:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:11:37.408654 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c57e09-2a88-4697-af74-9bda91a9ab48 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:11:37.468673 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:11:37.506135 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c57e09-2a88-4697-af74-9bda91a9ab48 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:11:37.537972 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93574524-fd01-4ec5-8199-762c945aa502 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:11:37.538565 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93574524-fd01-4ec5-8199-762c945aa502 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:11:37.538845 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93574524-fd01-4ec5-8199-762c945aa502 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:11:37.539113 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93574524-fd01-4ec5-8199-762c945aa502 None admin] PATH_INFO: `/v3/domains` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:11:37.539769 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93574524-fd01-4ec5-8199-762c945aa502 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:11:37.540067 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93574524-fd01-4ec5-8199-762c945aa502 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:11:37.541410 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93574524-fd01-4ec5-8199-762c945aa502 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:11:37.553995 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 610/1216] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Aug 24 09:11:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:11:37.559680 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f700a1-1054-4152-a090-b989847bc577 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:11:37.628066 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:11:37.657337 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f700a1-1054-4152-a090-b989847bc577 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:11:37.690531 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0573e80a-a89a-4c2c-ad7f-d9b2e23b6ece None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:11:37.691156 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0573e80a-a89a-4c2c-ad7f-d9b2e23b6ece None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:11:37.691433 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0573e80a-a89a-4c2c-ad7f-d9b2e23b6ece None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:11:37.691734 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0573e80a-a89a-4c2c-ad7f-d9b2e23b6ece None admin] PATH_INFO: `/v3/projects/service` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:11:37.698234 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0573e80a-a89a-4c2c-ad7f-d9b2e23b6ece None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:11:37.699675 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0573e80a-a89a-4c2c-ad7f-d9b2e23b6ece None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:11:37.711202 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-0573e80a-a89a-4c2c-ad7f-d9b2e23b6ece None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:11:37.711202 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:11:37.711202 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:11:37.711202 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:11:37.711202 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:11:37.711202 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:11:37.711202 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:11:37.711202 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:11:37.711202 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:11:37.711202 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:11:37.711202 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:11:37.711202 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:11:37.711202 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 09:11:37.711202 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 09:11:37.711202 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 09:11:37.712285 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 09:11:37.712285 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:11:37.712285 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:11:37.712285 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:11:37.712285 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:11:37.712285 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:11:37.712285 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:11:37.712285 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:11:37.712285 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 24 09:11:37.712285 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:11:37.712285 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:11:37.712285 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:11:37.712285 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:11:37.712285 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:11:37.712285 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:11:37.712285 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:11:37.712285 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:11:37.713204 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 09:11:37.713204 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 09:11:37.713204 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 09:11:37.713204 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 09:11:37.713204 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 09:11:37.713204 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 09:11:37.713204 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:11:37.713204 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 24 09:11:37.713204 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 607/1217] 10.222.0.16 () {56 vars in 1111 bytes} [Wed Aug 24 09:11:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 09:11:37.718197 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93574524-fd01-4ec5-8199-762c945aa502 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:11:37.819031 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93574524-fd01-4ec5-8199-762c945aa502 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:11:37.841588 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:11:37.853554 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40446170-25c5-4ca1-84fe-7cc3702132eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:11:37.854163 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40446170-25c5-4ca1-84fe-7cc3702132eb None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:11:37.854442 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40446170-25c5-4ca1-84fe-7cc3702132eb None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:11:37.854716 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40446170-25c5-4ca1-84fe-7cc3702132eb None admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:11:37.855562 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40446170-25c5-4ca1-84fe-7cc3702132eb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:11:37.855906 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40446170-25c5-4ca1-84fe-7cc3702132eb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:11:37.857720 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40446170-25c5-4ca1-84fe-7cc3702132eb None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:11:37.870138 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 611/1218] 10.222.0.16 () {56 vars in 1148 bytes} [Wed Aug 24 09:11:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:11:37.876429 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0573e80a-a89a-4c2c-ad7f-d9b2e23b6ece None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:11:37.977833 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0573e80a-a89a-4c2c-ad7f-d9b2e23b6ece None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:11:38.005792 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:11:38.011727 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754173f2-2214-4753-ad06-89cde85e6d90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:11:38.012329 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754173f2-2214-4753-ad06-89cde85e6d90 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:11:38.012642 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754173f2-2214-4753-ad06-89cde85e6d90 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:11:38.012913 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754173f2-2214-4753-ad06-89cde85e6d90 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:11:38.013632 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754173f2-2214-4753-ad06-89cde85e6d90 None admin] RBAC: Adding query filter params (role.id=67fd70a7d9794f7fb8faebffdec46dd2, scope.project.id=c467e88e2fb0491bacefcf6d2ce326d9, user.id=80cada22039d4f11935412ee578acd05) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:11:38.013942 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754173f2-2214-4753-ad06-89cde85e6d90 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:11:38.015352 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754173f2-2214-4753-ad06-89cde85e6d90 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:11:38.021098 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 608/1219] 10.222.0.16 () {56 vars in 1374 bytes} [Wed Aug 24 09:11:37 2022] GET /identity/v3/role_assignments?user.id=80cada22039d4f11935412ee578acd05&scope.project.id=c467e88e2fb0491bacefcf6d2ce326d9&role.id=67fd70a7d9794f7fb8faebffdec46dd2 => generated 259 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:11:39.905149 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a451546-abb4-475d-9c1b-1b213d28c119 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:11:39.905684 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a451546-abb4-475d-9c1b-1b213d28c119 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:11:39.905775 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a451546-abb4-475d-9c1b-1b213d28c119 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:11:39.907754 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 612/1220] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:11:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:11:39.913062 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f19b54-2feb-45ae-8edb-e9181eb9afd8 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:11:39.919539 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f19b54-2feb-45ae-8edb-e9181eb9afd8 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:11:39.919539 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f19b54-2feb-45ae-8edb-e9181eb9afd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:11:39.931190 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-72f19b54-2feb-45ae-8edb-e9181eb9afd8 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:11:39.932396 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-72f19b54-2feb-45ae-8edb-e9181eb9afd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:11:39.997837 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 609/1221] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:11:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 09:11:40.267665 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a451546-abb4-475d-9c1b-1b213d28c119 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:11:40.368080 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a451546-abb4-475d-9c1b-1b213d28c119 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:11:40.402383 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be727c2a-3d5b-409c-8b80-ef1253a6f9fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:11:40.403005 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be727c2a-3d5b-409c-8b80-ef1253a6f9fa None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:11:40.403283 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be727c2a-3d5b-409c-8b80-ef1253a6f9fa None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:11:40.403552 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be727c2a-3d5b-409c-8b80-ef1253a6f9fa None admin] PATH_INFO: `/v3/roles/service` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:11:40.413520 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be727c2a-3d5b-409c-8b80-ef1253a6f9fa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:11:40.418304 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-be727c2a-3d5b-409c-8b80-ef1253a6f9fa None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 09:11:40.418304 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:11:40.418304 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:11:40.418304 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:11:40.418304 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:11:40.418304 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:11:40.418304 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:11:40.418304 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:11:40.418304 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:11:40.418304 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:11:40.418304 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:11:40.418304 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:11:40.418304 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:11:40.418304 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:11:40.418304 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:11:40.418304 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:11:40.419395 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:11:40.419395 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:11:40.419395 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:11:40.419395 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:11:40.419395 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:11:40.419395 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:11:40.419395 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:11:40.419395 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:11:40.419395 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:11:40.419395 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:11:40.419395 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:11:40.419395 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Aug 24 09:11:40.419395 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:11:40.419395 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:11:40.419395 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:11:40.419395 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:11:40.420327 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:11:40.420327 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:11:40.420327 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:11:40.420327 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:11:40.420327 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:11:40.420327 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:11:40.420327 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:11:40.420327 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:11:40.420327 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:11:40.420327 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:11:40.420327 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 09:11:40.420327 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Aug 24 09:11:40.420327 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 613/1222] 10.222.0.16 () {56 vars in 1102 bytes} [Wed Aug 24 09:11:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:11:40.426154 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f19b54-2feb-45ae-8edb-e9181eb9afd8 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:11:40.522766 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f19b54-2feb-45ae-8edb-e9181eb9afd8 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:11:40.549804 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:11:40.560318 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d234a6-a244-40a8-a7e7-ad3bb9f81136 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:11:40.561532 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d234a6-a244-40a8-a7e7-ad3bb9f81136 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:11:40.561532 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d234a6-a244-40a8-a7e7-ad3bb9f81136 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:11:40.561532 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d234a6-a244-40a8-a7e7-ad3bb9f81136 None admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:11:40.562140 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d234a6-a244-40a8-a7e7-ad3bb9f81136 None admin] RBAC: Adding query filter params (name=service) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:11:40.562970 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d234a6-a244-40a8-a7e7-ad3bb9f81136 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:11:40.564347 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d234a6-a244-40a8-a7e7-ad3bb9f81136 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:11:40.572173 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 610/1223] 10.222.0.16 () {56 vars in 1103 bytes} [Wed Aug 24 09:11:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:11:40.578254 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be727c2a-3d5b-409c-8b80-ef1253a6f9fa None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:11:40.621539 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:11:40.675951 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be727c2a-3d5b-409c-8b80-ef1253a6f9fa None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:11:40.709505 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e32f792-d380-4a8a-9569-1b4babc4ad51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:11:40.710129 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e32f792-d380-4a8a-9569-1b4babc4ad51 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:11:40.710409 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e32f792-d380-4a8a-9569-1b4babc4ad51 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:11:40.710796 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e32f792-d380-4a8a-9569-1b4babc4ad51 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:11:40.721118 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e32f792-d380-4a8a-9569-1b4babc4ad51 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:11:40.722687 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e32f792-d380-4a8a-9569-1b4babc4ad51 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:11:40.736893 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-7e32f792-d380-4a8a-9569-1b4babc4ad51 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:11:40.736893 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:11:40.736893 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:11:40.736893 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:11:40.736893 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:11:40.736893 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:11:40.736893 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:11:40.736893 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:11:40.736893 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:11:40.736893 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:11:40.736893 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:11:40.736893 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:11:40.736893 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:11:40.736893 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:11:40.736893 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:11:40.736893 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:11:40.737998 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:11:40.737998 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:11:40.737998 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:11:40.737998 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:11:40.737998 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:11:40.737998 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:11:40.737998 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:11:40.737998 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Aug 24 09:11:40.737998 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:11:40.737998 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:11:40.737998 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:11:40.737998 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:11:40.737998 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:11:40.737998 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:11:40.737998 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:11:40.737998 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:11:40.737998 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:11:40.739227 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:11:40.739227 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:11:40.739227 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Aug 24 09:11:40.739227 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 614/1224] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 24 09:11:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:11:40.743754 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d234a6-a244-40a8-a7e7-ad3bb9f81136 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:11:40.845457 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d234a6-a244-40a8-a7e7-ad3bb9f81136 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:11:40.880024 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4670b7-5406-4112-9a7b-ba713418640f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:11:40.880628 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4670b7-5406-4112-9a7b-ba713418640f None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:11:40.880907 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4670b7-5406-4112-9a7b-ba713418640f None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:11:40.881180 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4670b7-5406-4112-9a7b-ba713418640f None admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:11:40.881929 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4670b7-5406-4112-9a7b-ba713418640f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:11:40.882240 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4670b7-5406-4112-9a7b-ba713418640f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:11:40.883620 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4670b7-5406-4112-9a7b-ba713418640f None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:11:40.895003 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 611/1225] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Aug 24 09:11:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:11:40.901001 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e32f792-d380-4a8a-9569-1b4babc4ad51 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:11:40.993930 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:11:41.000645 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e32f792-d380-4a8a-9569-1b4babc4ad51 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:11:41.033202 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d93ca75-6d2b-41a3-a6c8-bde6c966088b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:11:41.033823 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d93ca75-6d2b-41a3-a6c8-bde6c966088b None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:11:41.034103 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d93ca75-6d2b-41a3-a6c8-bde6c966088b None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:11:41.034378 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d93ca75-6d2b-41a3-a6c8-bde6c966088b None admin] PATH_INFO: `/v3/users/placement` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:11:41.041560 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d93ca75-6d2b-41a3-a6c8-bde6c966088b None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:11:41.043114 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d93ca75-6d2b-41a3-a6c8-bde6c966088b None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:11:41.054586 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-5d93ca75-6d2b-41a3-a6c8-bde6c966088b None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 09:11:41.054586 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:11:41.054586 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:11:41.054586 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:11:41.054586 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:11:41.054586 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:11:41.054586 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:11:41.054586 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:11:41.054586 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:11:41.054586 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:11:41.054586 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:11:41.054586 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:11:41.054586 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 09:11:41.054586 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 09:11:41.054586 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 09:11:41.054586 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 09:11:41.055752 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:11:41.055752 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:11:41.055752 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 09:11:41.055752 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:11:41.055752 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 09:11:41.055752 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:11:41.055752 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:11:41.055752 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:11:41.055752 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:11:41.055752 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:11:41.055752 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:11:41.055752 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Aug 24 09:11:41.055752 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:11:41.055752 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:11:41.055752 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:11:41.055752 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:11:41.055752 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:11:41.055752 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:11:41.057128 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:11:41.057128 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:11:41.057128 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 09:11:41.057128 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 09:11:41.057128 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 09:11:41.057128 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 09:11:41.057128 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 09:11:41.057128 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 09:11:41.057128 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 09:11:41.057128 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Aug 24 09:11:41.057128 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 615/1226] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 24 09:11:40 2022] GET /identity/v3/users/placement => generated 87 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:11:41.061693 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4670b7-5406-4112-9a7b-ba713418640f None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:11:41.109718 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:11:41.160049 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4670b7-5406-4112-9a7b-ba713418640f None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:11:41.193431 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f333108c-98e5-4ee3-8487-35725d96089e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:11:41.194074 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f333108c-98e5-4ee3-8487-35725d96089e None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:11:41.194347 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f333108c-98e5-4ee3-8487-35725d96089e None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:11:41.194617 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f333108c-98e5-4ee3-8487-35725d96089e None admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:11:41.195338 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f333108c-98e5-4ee3-8487-35725d96089e None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:11:41.195667 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f333108c-98e5-4ee3-8487-35725d96089e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:11:41.197080 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f333108c-98e5-4ee3-8487-35725d96089e None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:11:41.214338 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 612/1227] 10.222.0.16 () {56 vars in 1143 bytes} [Wed Aug 24 09:11:41 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:11:41.220773 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d93ca75-6d2b-41a3-a6c8-bde6c966088b None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:11:41.318509 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d93ca75-6d2b-41a3-a6c8-bde6c966088b None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:11:41.352692 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b53d590-b5b0-4465-96bf-3b3b901eebbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:11:41.353318 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b53d590-b5b0-4465-96bf-3b3b901eebbb None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:11:41.353597 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b53d590-b5b0-4465-96bf-3b3b901eebbb None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:11:41.353869 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b53d590-b5b0-4465-96bf-3b3b901eebbb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:11:41.364228 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b53d590-b5b0-4465-96bf-3b3b901eebbb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:11:41.365623 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b53d590-b5b0-4465-96bf-3b3b901eebbb None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:11:41.379198 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-2b53d590-b5b0-4465-96bf-3b3b901eebbb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:11:41.379198 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:11:41.379198 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:11:41.379198 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:11:41.379198 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:11:41.379198 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:11:41.379198 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:11:41.379198 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:11:41.379198 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:11:41.379198 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:11:41.379198 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:11:41.379198 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:11:41.379198 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:11:41.379198 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:11:41.379198 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:11:41.379198 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:11:41.383510 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:11:41.383510 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:11:41.383510 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:11:41.383510 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:11:41.383510 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:11:41.383510 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:11:41.383510 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:11:41.383510 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Aug 24 09:11:41.383510 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:11:41.383510 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:11:41.383510 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:11:41.383510 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:11:41.383510 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:11:41.383510 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:11:41.383510 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:11:41.383510 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:11:41.383510 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:11:41.384654 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:11:41.384654 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:11:41.384654 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Aug 24 09:11:41.384654 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 616/1228] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 24 09:11:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:11:41.386367 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f333108c-98e5-4ee3-8487-35725d96089e None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:11:41.486203 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f333108c-98e5-4ee3-8487-35725d96089e None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:11:41.488467 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:11:41.518642 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25de469b-90e1-42a1-bb52-7c57b8463f83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:11:41.519242 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25de469b-90e1-42a1-bb52-7c57b8463f83 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:11:41.519534 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25de469b-90e1-42a1-bb52-7c57b8463f83 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:11:41.519829 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25de469b-90e1-42a1-bb52-7c57b8463f83 None admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:11:41.520459 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25de469b-90e1-42a1-bb52-7c57b8463f83 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:11:41.520756 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25de469b-90e1-42a1-bb52-7c57b8463f83 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:11:41.522095 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25de469b-90e1-42a1-bb52-7c57b8463f83 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:11:41.533416 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 613/1229] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Aug 24 09:11:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:11:41.539178 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b53d590-b5b0-4465-96bf-3b3b901eebbb None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:11:41.548572 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:11:41.638435 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b53d590-b5b0-4465-96bf-3b3b901eebbb None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:11:41.671429 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad587aa3-0141-45fb-80b2-ae3efba5c99f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:11:41.672082 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad587aa3-0141-45fb-80b2-ae3efba5c99f None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:11:41.672365 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad587aa3-0141-45fb-80b2-ae3efba5c99f None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:11:41.672639 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad587aa3-0141-45fb-80b2-ae3efba5c99f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:11:41.679102 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad587aa3-0141-45fb-80b2-ae3efba5c99f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:11:41.680605 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad587aa3-0141-45fb-80b2-ae3efba5c99f None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:11:41.692078 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-ad587aa3-0141-45fb-80b2-ae3efba5c99f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:11:41.692078 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:11:41.692078 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:11:41.692078 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:11:41.692078 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:11:41.692078 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:11:41.692078 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:11:41.692078 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:11:41.692078 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:11:41.692078 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:11:41.692078 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:11:41.692078 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:11:41.692078 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 09:11:41.692078 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 09:11:41.692078 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 09:11:41.693104 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 09:11:41.693104 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:11:41.693104 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:11:41.693104 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:11:41.693104 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:11:41.693104 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:11:41.693104 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:11:41.693104 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:11:41.693104 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Aug 24 09:11:41.693104 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:11:41.693104 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:11:41.693104 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:11:41.693104 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:11:41.693104 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:11:41.693104 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:11:41.693104 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:11:41.693104 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:11:41.694151 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 09:11:41.694151 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 09:11:41.694151 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 09:11:41.694151 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 09:11:41.694151 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 09:11:41.694151 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 09:11:41.694151 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:11:41.694151 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Aug 24 09:11:41.694151 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 617/1230] 10.222.0.16 () {56 vars in 1111 bytes} [Wed Aug 24 09:11:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:11:41.698956 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25de469b-90e1-42a1-bb52-7c57b8463f83 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:11:41.796234 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25de469b-90e1-42a1-bb52-7c57b8463f83 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:11:41.829713 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d02a5cd-3c2e-43ce-9e69-d49f96df1473 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:11:41.830306 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d02a5cd-3c2e-43ce-9e69-d49f96df1473 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:11:41.830585 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d02a5cd-3c2e-43ce-9e69-d49f96df1473 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:11:41.830854 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d02a5cd-3c2e-43ce-9e69-d49f96df1473 None admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:11:41.831534 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d02a5cd-3c2e-43ce-9e69-d49f96df1473 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:11:41.831864 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d02a5cd-3c2e-43ce-9e69-d49f96df1473 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:11:41.833264 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d02a5cd-3c2e-43ce-9e69-d49f96df1473 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:11:41.844949 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 614/1231] 10.222.0.16 () {56 vars in 1148 bytes} [Wed Aug 24 09:11:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:11:41.851036 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad587aa3-0141-45fb-80b2-ae3efba5c99f None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:11:41.916705 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:11:41.953475 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad587aa3-0141-45fb-80b2-ae3efba5c99f None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:11:41.986115 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31df4bec-e77b-4745-bd58-c30d3a0af167 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:11:41.986755 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31df4bec-e77b-4745-bd58-c30d3a0af167 None admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:11:41.987029 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31df4bec-e77b-4745-bd58-c30d3a0af167 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:11:41.987298 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31df4bec-e77b-4745-bd58-c30d3a0af167 None admin] PATH_INFO: `/v3/projects/c467e88e2fb0491bacefcf6d2ce326d9/users/80cada22039d4f11935412ee578acd05/roles/67fd70a7d9794f7fb8faebffdec46dd2` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:11:42.001826 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31df4bec-e77b-4745-bd58-c30d3a0af167 None admin] RBAC: Authorizing `identity:create_grant(project_id=c467e88e2fb0491bacefcf6d2ce326d9, user_id=80cada22039d4f11935412ee578acd05, role_id=67fd70a7d9794f7fb8faebffdec46dd2)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:11:42.003251 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31df4bec-e77b-4745-bd58-c30d3a0af167 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:11:42.097961 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 618/1232] 10.222.0.16 () {58 vars in 1439 bytes} [Wed Aug 24 09:11:41 2022] PUT /identity/v3/projects/c467e88e2fb0491bacefcf6d2ce326d9/users/80cada22039d4f11935412ee578acd05/roles/67fd70a7d9794f7fb8faebffdec46dd2 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:11:43.991491 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e121f2a7-cc84-4731-ac2b-007272c5dc57 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:11:43.992113 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e121f2a7-cc84-4731-ac2b-007272c5dc57 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:11:43.992113 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e121f2a7-cc84-4731-ac2b-007272c5dc57 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:11:43.993416 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 615/1233] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:11:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:11:43.998661 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3796a4-3d67-4ad7-97c9-ce4c41f3259b None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:11:44.001422 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3796a4-3d67-4ad7-97c9-ce4c41f3259b None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:11:44.001422 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3796a4-3d67-4ad7-97c9-ce4c41f3259b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:11:44.016662 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-df3796a4-3d67-4ad7-97c9-ce4c41f3259b None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:11:44.017837 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-df3796a4-3d67-4ad7-97c9-ce4c41f3259b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:11:44.083568 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 619/1234] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:11:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 09:11:44.353976 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e121f2a7-cc84-4731-ac2b-007272c5dc57 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:11:44.378924 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:11:44.454937 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e121f2a7-cc84-4731-ac2b-007272c5dc57 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:11:44.488173 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d0a899-710b-4c08-b583-05483f99a602 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:11:44.493908 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d0a899-710b-4c08-b583-05483f99a602 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:11:44.493908 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d0a899-710b-4c08-b583-05483f99a602 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:11:44.493908 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d0a899-710b-4c08-b583-05483f99a602 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:11:44.499323 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d0a899-710b-4c08-b583-05483f99a602 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:11:44.504013 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-b4d0a899-710b-4c08-b583-05483f99a602 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 09:11:44.504013 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:11:44.504013 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:11:44.504013 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:11:44.504013 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:11:44.504013 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:11:44.504013 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:11:44.504013 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:11:44.504013 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:11:44.504013 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:11:44.504013 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:11:44.504013 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:11:44.504013 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:11:44.504013 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:11:44.504013 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:11:44.504013 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:11:44.505220 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:11:44.505220 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:11:44.505220 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:11:44.505220 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:11:44.505220 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:11:44.505220 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:11:44.505220 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:11:44.505220 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:11:44.505220 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:11:44.505220 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:11:44.505220 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:11:44.505220 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 24 09:11:44.505220 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:11:44.505220 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:11:44.505220 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:11:44.505220 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:11:44.506404 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:11:44.506404 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:11:44.506404 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:11:44.506404 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:11:44.506404 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:11:44.506404 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:11:44.506404 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:11:44.506404 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:11:44.506404 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:11:44.506404 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:11:44.506404 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 09:11:44.506404 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 24 09:11:44.506404 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 616/1235] 10.222.0.16 () {56 vars in 1102 bytes} [Wed Aug 24 09:11:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:11:44.511083 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3796a4-3d67-4ad7-97c9-ce4c41f3259b None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:11:44.557898 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:11:44.611318 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3796a4-3d67-4ad7-97c9-ce4c41f3259b None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:11:44.644954 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45f85d7-08c0-4324-8708-046c4bbd7432 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:11:44.645552 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45f85d7-08c0-4324-8708-046c4bbd7432 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:11:44.645834 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45f85d7-08c0-4324-8708-046c4bbd7432 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:11:44.646109 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45f85d7-08c0-4324-8708-046c4bbd7432 None admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:11:44.646774 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45f85d7-08c0-4324-8708-046c4bbd7432 None admin] RBAC: Adding query filter params (name=service) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:11:44.647080 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45f85d7-08c0-4324-8708-046c4bbd7432 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:11:44.648524 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45f85d7-08c0-4324-8708-046c4bbd7432 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:11:44.657137 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 620/1236] 10.222.0.16 () {56 vars in 1103 bytes} [Wed Aug 24 09:11:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:11:44.662903 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d0a899-710b-4c08-b583-05483f99a602 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:11:44.745358 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:11:44.761988 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d0a899-710b-4c08-b583-05483f99a602 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:11:44.796676 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab23d799-8717-4159-a07a-baafd2fc68dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:11:44.797302 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab23d799-8717-4159-a07a-baafd2fc68dd None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:11:44.797577 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab23d799-8717-4159-a07a-baafd2fc68dd None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:11:44.797846 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab23d799-8717-4159-a07a-baafd2fc68dd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:11:44.807431 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab23d799-8717-4159-a07a-baafd2fc68dd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:11:44.809133 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab23d799-8717-4159-a07a-baafd2fc68dd None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:11:44.823382 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-ab23d799-8717-4159-a07a-baafd2fc68dd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:11:44.823382 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:11:44.823382 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:11:44.823382 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:11:44.823382 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:11:44.823382 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:11:44.823382 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:11:44.823382 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:11:44.823382 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:11:44.823382 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:11:44.823382 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:11:44.823382 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:11:44.823382 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:11:44.823382 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:11:44.823382 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:11:44.823382 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:11:44.824543 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:11:44.824543 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:11:44.824543 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:11:44.824543 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:11:44.824543 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:11:44.824543 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:11:44.824543 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:11:44.824543 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 24 09:11:44.824543 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:11:44.824543 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:11:44.824543 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:11:44.824543 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:11:44.824543 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:11:44.824543 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:11:44.824543 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:11:44.824543 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:11:44.824543 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:11:44.825522 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:11:44.825522 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:11:44.825522 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 24 09:11:44.825522 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 617/1237] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 24 09:11:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:11:44.831222 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45f85d7-08c0-4324-8708-046c4bbd7432 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:11:44.930594 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45f85d7-08c0-4324-8708-046c4bbd7432 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:11:44.940298 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:11:44.964593 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f0d11f-87ff-4b80-96df-7f8fbe707272 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:11:44.965236 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f0d11f-87ff-4b80-96df-7f8fbe707272 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:11:44.965513 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f0d11f-87ff-4b80-96df-7f8fbe707272 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:11:44.965781 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f0d11f-87ff-4b80-96df-7f8fbe707272 None admin] PATH_INFO: `/v3/domains` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:11:44.966407 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f0d11f-87ff-4b80-96df-7f8fbe707272 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:11:44.966707 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f0d11f-87ff-4b80-96df-7f8fbe707272 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:11:44.968101 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f0d11f-87ff-4b80-96df-7f8fbe707272 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:11:44.979471 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 621/1238] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Aug 24 09:11:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:11:44.985823 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab23d799-8717-4159-a07a-baafd2fc68dd None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:11:45.083176 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab23d799-8717-4159-a07a-baafd2fc68dd None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:11:45.115536 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ee7a17-a24f-4beb-ace2-493a12ceaf7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:11:45.116178 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ee7a17-a24f-4beb-ace2-493a12ceaf7c None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:11:45.116448 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ee7a17-a24f-4beb-ace2-493a12ceaf7c None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:11:45.116713 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ee7a17-a24f-4beb-ace2-493a12ceaf7c None admin] PATH_INFO: `/v3/users/placement` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:11:45.119274 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:11:45.124115 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ee7a17-a24f-4beb-ace2-493a12ceaf7c None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:11:45.126158 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ee7a17-a24f-4beb-ace2-493a12ceaf7c None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:11:45.136968 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-c4ee7a17-a24f-4beb-ace2-493a12ceaf7c None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 09:11:45.136968 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:11:45.136968 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:11:45.136968 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:11:45.136968 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:11:45.136968 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:11:45.136968 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:11:45.136968 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:11:45.136968 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:11:45.136968 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:11:45.136968 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:11:45.136968 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:11:45.136968 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 09:11:45.136968 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 09:11:45.136968 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 09:11:45.136968 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 09:11:45.138098 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:11:45.138098 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:11:45.138098 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 09:11:45.138098 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:11:45.138098 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 09:11:45.138098 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:11:45.138098 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:11:45.138098 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:11:45.138098 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:11:45.138098 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:11:45.138098 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:11:45.138098 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 24 09:11:45.138098 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:11:45.138098 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:11:45.138098 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:11:45.138098 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:11:45.138098 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:11:45.138098 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:11:45.139343 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:11:45.139343 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:11:45.139343 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 09:11:45.139343 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 09:11:45.139343 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 09:11:45.139343 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 09:11:45.139343 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 09:11:45.139343 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 09:11:45.139343 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 09:11:45.139343 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 24 09:11:45.142875 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 618/1239] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 24 09:11:44 2022] GET /identity/v3/users/placement => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:11:45.143901 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f0d11f-87ff-4b80-96df-7f8fbe707272 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:11:45.241079 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f0d11f-87ff-4b80-96df-7f8fbe707272 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:11:45.276187 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0848d1c-d623-4a31-bb46-b6ed994f8bb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:11:45.276807 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0848d1c-d623-4a31-bb46-b6ed994f8bb5 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:11:45.277089 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0848d1c-d623-4a31-bb46-b6ed994f8bb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:11:45.277363 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0848d1c-d623-4a31-bb46-b6ed994f8bb5 None admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:11:45.278142 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0848d1c-d623-4a31-bb46-b6ed994f8bb5 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:11:45.278451 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0848d1c-d623-4a31-bb46-b6ed994f8bb5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:11:45.279908 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0848d1c-d623-4a31-bb46-b6ed994f8bb5 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:11:45.300837 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 622/1240] 10.222.0.16 () {56 vars in 1143 bytes} [Wed Aug 24 09:11:45 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:11:45.305712 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ee7a17-a24f-4beb-ace2-493a12ceaf7c None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:11:45.404696 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ee7a17-a24f-4beb-ace2-493a12ceaf7c None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:11:45.438000 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522c02ed-3b88-4306-8afa-dd57a7e2e5f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:11:45.438596 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522c02ed-3b88-4306-8afa-dd57a7e2e5f5 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:11:45.438874 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522c02ed-3b88-4306-8afa-dd57a7e2e5f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:11:45.439188 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522c02ed-3b88-4306-8afa-dd57a7e2e5f5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:11:45.499927 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:11:45.506280 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522c02ed-3b88-4306-8afa-dd57a7e2e5f5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:11:45.507750 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522c02ed-3b88-4306-8afa-dd57a7e2e5f5 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:11:45.606087 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 619/1241] 10.222.0.16 () {58 vars in 1282 bytes} [Wed Aug 24 09:11:45 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 09:11:45.612582 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0848d1c-d623-4a31-bb46-b6ed994f8bb5 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:11:45.639232 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:11:45.710893 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0848d1c-d623-4a31-bb46-b6ed994f8bb5 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:11:45.744236 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170b89ef-d1a5-41d0-8f53-900f79e48c8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:11:45.744853 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170b89ef-d1a5-41d0-8f53-900f79e48c8c None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:11:45.745130 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170b89ef-d1a5-41d0-8f53-900f79e48c8c None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:11:45.745435 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170b89ef-d1a5-41d0-8f53-900f79e48c8c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:11:45.755080 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170b89ef-d1a5-41d0-8f53-900f79e48c8c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:11:45.756547 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170b89ef-d1a5-41d0-8f53-900f79e48c8c None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:11:45.772065 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-170b89ef-d1a5-41d0-8f53-900f79e48c8c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:11:45.772065 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:11:45.772065 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:11:45.772065 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:11:45.772065 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:11:45.772065 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:11:45.772065 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:11:45.772065 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:11:45.772065 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:11:45.772065 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:11:45.772065 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:11:45.772065 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:11:45.772065 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:11:45.772065 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:11:45.772065 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:11:45.772065 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:11:45.773067 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:11:45.773067 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:11:45.773067 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:11:45.773067 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:11:45.773067 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:11:45.773067 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:11:45.773067 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:11:45.773067 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Aug 24 09:11:45.773067 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:11:45.773067 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:11:45.773067 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:11:45.773067 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:11:45.773067 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:11:45.773067 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:11:45.773067 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:11:45.773067 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:11:45.773067 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:11:45.773999 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:11:45.773999 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:11:45.773999 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Aug 24 09:11:45.773999 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 623/1242] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 24 09:11:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:11:45.778808 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522c02ed-3b88-4306-8afa-dd57a7e2e5f5 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:11:45.871477 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:11:45.877239 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522c02ed-3b88-4306-8afa-dd57a7e2e5f5 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:11:45.911096 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b793530-b98e-438c-b3d7-2985798e0623 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:11:45.911730 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b793530-b98e-438c-b3d7-2985798e0623 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:11:45.912010 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b793530-b98e-438c-b3d7-2985798e0623 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:11:45.912283 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b793530-b98e-438c-b3d7-2985798e0623 None admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:11:45.912962 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b793530-b98e-438c-b3d7-2985798e0623 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:11:45.919487 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b793530-b98e-438c-b3d7-2985798e0623 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:11:45.919487 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b793530-b98e-438c-b3d7-2985798e0623 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:11:45.925480 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 620/1243] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Aug 24 09:11:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:11:45.931157 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170b89ef-d1a5-41d0-8f53-900f79e48c8c None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:11:46.008815 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:11:46.029753 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170b89ef-d1a5-41d0-8f53-900f79e48c8c None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:11:46.062614 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991eac8a-da8b-47cc-b00b-da096fb504ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:11:46.071889 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991eac8a-da8b-47cc-b00b-da096fb504ef None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:11:46.071889 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991eac8a-da8b-47cc-b00b-da096fb504ef None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:11:46.071889 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991eac8a-da8b-47cc-b00b-da096fb504ef None admin] PATH_INFO: `/v3/projects/service` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:11:46.071889 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991eac8a-da8b-47cc-b00b-da096fb504ef None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:11:46.072357 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991eac8a-da8b-47cc-b00b-da096fb504ef None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:11:46.084333 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-991eac8a-da8b-47cc-b00b-da096fb504ef None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:11:46.084333 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:11:46.084333 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:11:46.084333 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:11:46.084333 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:11:46.084333 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:11:46.084333 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:11:46.084333 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:11:46.084333 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:11:46.084333 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:11:46.084333 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:11:46.084333 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:11:46.084333 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 09:11:46.084333 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 09:11:46.084333 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 09:11:46.085448 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 09:11:46.085448 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:11:46.085448 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:11:46.085448 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:11:46.085448 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:11:46.085448 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:11:46.085448 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:11:46.085448 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:11:46.085448 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Aug 24 09:11:46.085448 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:11:46.085448 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:11:46.085448 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:11:46.085448 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:11:46.085448 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:11:46.085448 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:11:46.085448 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:11:46.085448 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:11:46.086565 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 09:11:46.086565 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 09:11:46.086565 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 09:11:46.086565 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 09:11:46.086565 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 09:11:46.086565 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 09:11:46.086565 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:11:46.086565 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Aug 24 09:11:46.086565 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 624/1244] 10.222.0.16 () {56 vars in 1111 bytes} [Wed Aug 24 09:11:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:11:46.091316 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b793530-b98e-438c-b3d7-2985798e0623 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:11:46.190343 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b793530-b98e-438c-b3d7-2985798e0623 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:11:46.223897 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde5c75e-6739-4783-a111-5c22f3183d6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:11:46.224503 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde5c75e-6739-4783-a111-5c22f3183d6e None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:11:46.224781 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde5c75e-6739-4783-a111-5c22f3183d6e None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:11:46.225164 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde5c75e-6739-4783-a111-5c22f3183d6e None admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:11:46.225861 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde5c75e-6739-4783-a111-5c22f3183d6e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:11:46.226166 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde5c75e-6739-4783-a111-5c22f3183d6e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:11:46.227607 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde5c75e-6739-4783-a111-5c22f3183d6e None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:11:46.239316 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 621/1245] 10.222.0.16 () {56 vars in 1148 bytes} [Wed Aug 24 09:11:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:11:46.245324 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991eac8a-da8b-47cc-b00b-da096fb504ef None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:11:46.346548 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991eac8a-da8b-47cc-b00b-da096fb504ef None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:11:46.379218 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af0fc0b-3d10-4f52-8a0a-ae123b26b1a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:11:46.379841 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af0fc0b-3d10-4f52-8a0a-ae123b26b1a3 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:11:46.380118 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af0fc0b-3d10-4f52-8a0a-ae123b26b1a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:11:46.380388 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af0fc0b-3d10-4f52-8a0a-ae123b26b1a3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:11:46.381103 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af0fc0b-3d10-4f52-8a0a-ae123b26b1a3 None admin] RBAC: Adding query filter params (role.id=67fd70a7d9794f7fb8faebffdec46dd2, scope.project.id=c467e88e2fb0491bacefcf6d2ce326d9, user.id=80cada22039d4f11935412ee578acd05) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:11:46.381409 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af0fc0b-3d10-4f52-8a0a-ae123b26b1a3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:11:46.382861 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af0fc0b-3d10-4f52-8a0a-ae123b26b1a3 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:11:46.388637 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 625/1246] 10.222.0.16 () {56 vars in 1374 bytes} [Wed Aug 24 09:11:46 2022] GET /identity/v3/role_assignments?user.id=80cada22039d4f11935412ee578acd05&scope.project.id=c467e88e2fb0491bacefcf6d2ce326d9&role.id=67fd70a7d9794f7fb8faebffdec46dd2 => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:11:48.359714 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0461de73-6ddd-41cc-840b-6af35847d61d None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:11:48.360247 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0461de73-6ddd-41cc-840b-6af35847d61d None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:11:48.360247 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0461de73-6ddd-41cc-840b-6af35847d61d None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:11:48.361595 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 622/1247] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:11:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:11:48.366820 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ae397c-65c3-41b2-bbe1-49d280f056d3 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:11:48.367201 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ae397c-65c3-41b2-bbe1-49d280f056d3 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:11:48.367383 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ae397c-65c3-41b2-bbe1-49d280f056d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:11:48.373745 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:11:48.386405 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-b0ae397c-65c3-41b2-bbe1-49d280f056d3 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:11:48.387671 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-b0ae397c-65c3-41b2-bbe1-49d280f056d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:11:48.452036 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 626/1248] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:11:48 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 09:11:48.726678 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0461de73-6ddd-41cc-840b-6af35847d61d None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:11:48.745124 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:11:48.825099 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0461de73-6ddd-41cc-840b-6af35847d61d None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:11:48.858861 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7f82b7-764b-48a7-9265-f5f6410b6610 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:11:48.860059 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7f82b7-764b-48a7-9265-f5f6410b6610 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:11:48.860339 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7f82b7-764b-48a7-9265-f5f6410b6610 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:11:48.860610 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7f82b7-764b-48a7-9265-f5f6410b6610 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:11:48.870410 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7f82b7-764b-48a7-9265-f5f6410b6610 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:11:48.875777 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-8f7f82b7-764b-48a7-9265-f5f6410b6610 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:11:48.875777 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:11:48.875777 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:11:48.875777 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:11:48.875777 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:11:48.875777 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:11:48.875777 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:11:48.875777 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:11:48.875777 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:11:48.875777 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:11:48.875777 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:11:48.875777 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:11:48.875777 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:11:48.875777 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:11:48.875777 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:11:48.875777 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:11:48.876896 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:11:48.876896 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:11:48.876896 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:11:48.876896 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:11:48.876896 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:11:48.876896 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:11:48.876896 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:11:48.876896 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:11:48.876896 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:11:48.876896 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:11:48.876896 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:11:48.876896 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 24 09:11:48.876896 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:11:48.876896 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:11:48.876896 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:11:48.876896 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:11:48.877862 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:11:48.877862 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:11:48.877862 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:11:48.877862 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:11:48.877862 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:11:48.877862 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:11:48.877862 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:11:48.877862 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:11:48.877862 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:11:48.877862 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:11:48.877862 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:11:48.877862 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 24 09:11:48.877862 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 623/1249] 10.222.0.16 () {56 vars in 1096 bytes} [Wed Aug 24 09:11:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:11:48.883140 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ae397c-65c3-41b2-bbe1-49d280f056d3 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:11:48.990427 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ae397c-65c3-41b2-bbe1-49d280f056d3 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:11:49.020528 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:11:49.025240 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbe0455-2d87-46b1-9194-3115163af348 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:11:49.025841 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbe0455-2d87-46b1-9194-3115163af348 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:11:49.026118 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbe0455-2d87-46b1-9194-3115163af348 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:11:49.026387 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbe0455-2d87-46b1-9194-3115163af348 None admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:11:49.027038 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbe0455-2d87-46b1-9194-3115163af348 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:11:49.027337 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbe0455-2d87-46b1-9194-3115163af348 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:11:49.028729 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbe0455-2d87-46b1-9194-3115163af348 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:11:49.037001 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 627/1250] 10.222.0.16 () {56 vars in 1099 bytes} [Wed Aug 24 09:11:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:11:49.044434 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7f82b7-764b-48a7-9265-f5f6410b6610 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:11:49.119631 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:11:49.141168 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7f82b7-764b-48a7-9265-f5f6410b6610 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:11:49.174174 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d78b26-bf7e-41b6-bae0-cc14dd1cc505 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:11:49.174742 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d78b26-bf7e-41b6-bae0-cc14dd1cc505 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:11:49.175012 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d78b26-bf7e-41b6-bae0-cc14dd1cc505 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:11:49.175344 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d78b26-bf7e-41b6-bae0-cc14dd1cc505 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:11:49.185393 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d78b26-bf7e-41b6-bae0-cc14dd1cc505 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:11:49.186829 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d78b26-bf7e-41b6-bae0-cc14dd1cc505 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:11:49.200730 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-42d78b26-bf7e-41b6-bae0-cc14dd1cc505 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:11:49.200730 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:11:49.200730 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:11:49.200730 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:11:49.200730 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:11:49.200730 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:11:49.200730 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:11:49.200730 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:11:49.200730 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:11:49.200730 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:11:49.200730 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:11:49.200730 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:11:49.200730 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:11:49.200730 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:11:49.200730 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:11:49.200730 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:11:49.202166 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:11:49.202166 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:11:49.202166 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:11:49.202166 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:11:49.202166 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:11:49.202166 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:11:49.202166 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:11:49.202166 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 24 09:11:49.202166 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:11:49.202166 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:11:49.202166 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:11:49.202166 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:11:49.202166 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:11:49.202166 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:11:49.202166 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:11:49.202166 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:11:49.202166 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:11:49.203289 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:11:49.203289 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:11:49.203289 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 24 09:11:49.203289 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 624/1251] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 24 09:11:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:11:49.208257 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbe0455-2d87-46b1-9194-3115163af348 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:11:49.312413 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbe0455-2d87-46b1-9194-3115163af348 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:11:49.346552 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c734ee-76f7-41fd-95a0-447f2cf48591 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:11:49.347164 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c734ee-76f7-41fd-95a0-447f2cf48591 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:11:49.347445 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c734ee-76f7-41fd-95a0-447f2cf48591 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:11:49.347758 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c734ee-76f7-41fd-95a0-447f2cf48591 None admin] PATH_INFO: `/v3/domains` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:11:49.348399 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c734ee-76f7-41fd-95a0-447f2cf48591 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:11:49.348701 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c734ee-76f7-41fd-95a0-447f2cf48591 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:11:49.350065 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c734ee-76f7-41fd-95a0-447f2cf48591 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:11:49.362332 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 628/1252] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Aug 24 09:11:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:11:49.368105 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d78b26-bf7e-41b6-bae0-cc14dd1cc505 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:11:49.465793 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d78b26-bf7e-41b6-bae0-cc14dd1cc505 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:11:49.491496 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:11:49.499025 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e660a3-7fe2-4ca6-a688-436df1a9df00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:11:49.501576 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e660a3-7fe2-4ca6-a688-436df1a9df00 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:11:49.501576 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e660a3-7fe2-4ca6-a688-436df1a9df00 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:11:49.501576 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e660a3-7fe2-4ca6-a688-436df1a9df00 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:11:49.508272 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e660a3-7fe2-4ca6-a688-436df1a9df00 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:11:49.509685 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e660a3-7fe2-4ca6-a688-436df1a9df00 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:11:49.520348 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-95e660a3-7fe2-4ca6-a688-436df1a9df00 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 09:11:49.520348 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:11:49.520348 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:11:49.520348 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:11:49.520348 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:11:49.520348 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:11:49.520348 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:11:49.520348 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:11:49.520348 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:11:49.520348 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:11:49.520348 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:11:49.520348 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:11:49.520348 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 09:11:49.520348 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 09:11:49.520348 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 09:11:49.520348 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 09:11:49.521496 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:11:49.521496 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:11:49.521496 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 09:11:49.521496 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:11:49.521496 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 09:11:49.521496 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:11:49.521496 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:11:49.521496 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:11:49.521496 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:11:49.521496 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:11:49.521496 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:11:49.521496 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 24 09:11:49.521496 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:11:49.521496 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:11:49.521496 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:11:49.521496 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:11:49.521496 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:11:49.521496 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:11:49.522489 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:11:49.522489 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:11:49.522489 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 09:11:49.522489 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 09:11:49.522489 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 09:11:49.522489 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 09:11:49.522489 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 09:11:49.522489 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 09:11:49.522489 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 09:11:49.522489 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 24 09:11:49.522489 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 625/1253] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 24 09:11:49 2022] GET /identity/v3/users/placement => generated 87 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:11:49.527291 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c734ee-76f7-41fd-95a0-447f2cf48591 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:11:49.581369 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:11:49.640341 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c734ee-76f7-41fd-95a0-447f2cf48591 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:11:49.674194 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ca2f29-bf7b-4586-82fa-a33c1773ddee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:11:49.674791 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ca2f29-bf7b-4586-82fa-a33c1773ddee None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:11:49.675068 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ca2f29-bf7b-4586-82fa-a33c1773ddee None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:11:49.675338 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ca2f29-bf7b-4586-82fa-a33c1773ddee None admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:11:49.676106 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ca2f29-bf7b-4586-82fa-a33c1773ddee None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:11:49.676409 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ca2f29-bf7b-4586-82fa-a33c1773ddee None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:11:49.677821 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ca2f29-bf7b-4586-82fa-a33c1773ddee None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:11:49.694950 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 629/1254] 10.222.0.16 () {56 vars in 1143 bytes} [Wed Aug 24 09:11:49 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:11:49.700758 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e660a3-7fe2-4ca6-a688-436df1a9df00 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:11:49.802523 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e660a3-7fe2-4ca6-a688-436df1a9df00 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:11:49.836442 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1ba5db-57f4-4ce5-875e-1ea26cae260d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:11:49.836977 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1ba5db-57f4-4ce5-875e-1ea26cae260d None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:11:49.837256 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1ba5db-57f4-4ce5-875e-1ea26cae260d None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:11:49.837525 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1ba5db-57f4-4ce5-875e-1ea26cae260d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:11:49.889819 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:11:49.902541 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1ba5db-57f4-4ce5-875e-1ea26cae260d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:11:49.903999 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1ba5db-57f4-4ce5-875e-1ea26cae260d None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:11:49.996722 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 626/1255] 10.222.0.16 () {58 vars in 1282 bytes} [Wed Aug 24 09:11:49 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 09:11:50.003340 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ca2f29-bf7b-4586-82fa-a33c1773ddee None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:11:50.101355 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ca2f29-bf7b-4586-82fa-a33c1773ddee None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:11:50.114456 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:11:50.134721 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b485cc15-f0b9-4e40-8e3e-649df56f5877 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:11:50.135339 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b485cc15-f0b9-4e40-8e3e-649df56f5877 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:11:50.135642 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b485cc15-f0b9-4e40-8e3e-649df56f5877 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:11:50.135955 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b485cc15-f0b9-4e40-8e3e-649df56f5877 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:11:50.146160 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b485cc15-f0b9-4e40-8e3e-649df56f5877 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:11:50.147644 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b485cc15-f0b9-4e40-8e3e-649df56f5877 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:11:50.161601 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-b485cc15-f0b9-4e40-8e3e-649df56f5877 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:11:50.161601 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:11:50.161601 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:11:50.161601 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:11:50.161601 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:11:50.161601 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:11:50.161601 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:11:50.161601 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:11:50.161601 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:11:50.161601 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:11:50.161601 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:11:50.161601 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:11:50.161601 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:11:50.161601 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:11:50.161601 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:11:50.161601 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:11:50.162787 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:11:50.162787 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:11:50.162787 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:11:50.162787 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:11:50.162787 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:11:50.162787 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:11:50.162787 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:11:50.162787 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Aug 24 09:11:50.162787 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:11:50.162787 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:11:50.162787 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:11:50.162787 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:11:50.162787 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:11:50.162787 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:11:50.162787 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:11:50.162787 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:11:50.162787 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:11:50.163961 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:11:50.163961 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:11:50.163961 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Aug 24 09:11:50.163961 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 630/1256] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 24 09:11:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:11:50.168589 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1ba5db-57f4-4ce5-875e-1ea26cae260d None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:11:50.256307 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:11:50.267911 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1ba5db-57f4-4ce5-875e-1ea26cae260d None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:11:50.301279 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d36782-d88d-4fee-88c0-a3683131fbfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:11:50.302267 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d36782-d88d-4fee-88c0-a3683131fbfe None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:11:50.302267 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d36782-d88d-4fee-88c0-a3683131fbfe None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:11:50.302564 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d36782-d88d-4fee-88c0-a3683131fbfe None admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:11:50.303469 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d36782-d88d-4fee-88c0-a3683131fbfe None admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:11:50.303469 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d36782-d88d-4fee-88c0-a3683131fbfe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:11:50.304722 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d36782-d88d-4fee-88c0-a3683131fbfe None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:11:50.316560 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 627/1257] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Aug 24 09:11:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:11:50.322235 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b485cc15-f0b9-4e40-8e3e-649df56f5877 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:11:50.420484 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b485cc15-f0b9-4e40-8e3e-649df56f5877 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:11:50.454439 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5667586-b767-48e8-9059-faed7dba423a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:11:50.455063 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5667586-b767-48e8-9059-faed7dba423a None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:11:50.455348 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5667586-b767-48e8-9059-faed7dba423a None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:11:50.455648 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5667586-b767-48e8-9059-faed7dba423a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:11:50.462141 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5667586-b767-48e8-9059-faed7dba423a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:11:50.463606 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5667586-b767-48e8-9059-faed7dba423a None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:11:50.475384 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-e5667586-b767-48e8-9059-faed7dba423a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:11:50.475384 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:11:50.475384 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:11:50.475384 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:11:50.475384 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:11:50.475384 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:11:50.475384 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:11:50.475384 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:11:50.475384 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:11:50.475384 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:11:50.475384 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:11:50.475384 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:11:50.475384 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 09:11:50.475384 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 09:11:50.475384 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 09:11:50.476553 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 09:11:50.476553 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:11:50.476553 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:11:50.476553 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:11:50.476553 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:11:50.476553 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:11:50.476553 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:11:50.476553 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:11:50.476553 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Aug 24 09:11:50.476553 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:11:50.476553 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:11:50.476553 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:11:50.476553 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:11:50.476553 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:11:50.476553 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:11:50.476553 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:11:50.476553 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:11:50.477642 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 09:11:50.477642 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 09:11:50.477642 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 09:11:50.477642 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 09:11:50.477642 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 09:11:50.477642 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 09:11:50.477642 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:11:50.477642 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Aug 24 09:11:50.477642 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 631/1258] 10.222.0.16 () {56 vars in 1111 bytes} [Wed Aug 24 09:11:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:11:50.482404 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d36782-d88d-4fee-88c0-a3683131fbfe None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:11:50.584478 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d36782-d88d-4fee-88c0-a3683131fbfe None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:11:50.617301 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e105645c-d356-4310-bebb-e77ac96fc9f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:11:50.617904 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e105645c-d356-4310-bebb-e77ac96fc9f5 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:11:50.618177 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e105645c-d356-4310-bebb-e77ac96fc9f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:11:50.618445 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e105645c-d356-4310-bebb-e77ac96fc9f5 None admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:11:50.619127 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e105645c-d356-4310-bebb-e77ac96fc9f5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:11:50.619428 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e105645c-d356-4310-bebb-e77ac96fc9f5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:11:50.620891 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e105645c-d356-4310-bebb-e77ac96fc9f5 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:11:50.632953 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 628/1259] 10.222.0.16 () {56 vars in 1148 bytes} [Wed Aug 24 09:11:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:11:50.639106 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5667586-b767-48e8-9059-faed7dba423a None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:11:50.667275 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:11:50.738977 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5667586-b767-48e8-9059-faed7dba423a None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:11:50.772780 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb52450-73ec-4de2-af8b-f58e29a29a8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:11:50.773399 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb52450-73ec-4de2-af8b-f58e29a29a8e None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:11:50.773675 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb52450-73ec-4de2-af8b-f58e29a29a8e None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:11:50.773943 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb52450-73ec-4de2-af8b-f58e29a29a8e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:11:50.774654 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb52450-73ec-4de2-af8b-f58e29a29a8e None admin] RBAC: Adding query filter params (role.id=15f634d4a2d247ddab90c458e5e1c141, scope.project.id=c467e88e2fb0491bacefcf6d2ce326d9, user.id=80cada22039d4f11935412ee578acd05) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:11:50.774958 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb52450-73ec-4de2-af8b-f58e29a29a8e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:11:50.776435 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb52450-73ec-4de2-af8b-f58e29a29a8e None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:11:50.781693 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 632/1260] 10.222.0.16 () {56 vars in 1374 bytes} [Wed Aug 24 09:11:50 2022] GET /identity/v3/role_assignments?user.id=80cada22039d4f11935412ee578acd05&scope.project.id=c467e88e2fb0491bacefcf6d2ce326d9&role.id=15f634d4a2d247ddab90c458e5e1c141 => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:11:52.641788 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa69274-1f80-4826-8f77-55eb613a3daa None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:11:52.642332 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa69274-1f80-4826-8f77-55eb613a3daa None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:11:52.642332 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa69274-1f80-4826-8f77-55eb613a3daa None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:11:52.643604 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 629/1261] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:11:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:11:52.648866 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77c2293-e29a-47f7-ac9a-e0ee5fe35530 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:11:52.649279 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77c2293-e29a-47f7-ac9a-e0ee5fe35530 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:11:52.649759 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77c2293-e29a-47f7-ac9a-e0ee5fe35530 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:11:52.668047 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-b77c2293-e29a-47f7-ac9a-e0ee5fe35530 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:11:52.669237 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-b77c2293-e29a-47f7-ac9a-e0ee5fe35530 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:11:52.735104 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 633/1262] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:11:52 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 09:11:53.008562 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa69274-1f80-4826-8f77-55eb613a3daa None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:11:53.019931 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:11:53.108075 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa69274-1f80-4826-8f77-55eb613a3daa None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:11:53.141036 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d275bed3-4d17-45ca-b988-a575fc4a4a96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:11:53.141651 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d275bed3-4d17-45ca-b988-a575fc4a4a96 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:11:53.141965 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d275bed3-4d17-45ca-b988-a575fc4a4a96 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:11:53.142238 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d275bed3-4d17-45ca-b988-a575fc4a4a96 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:11:53.152146 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d275bed3-4d17-45ca-b988-a575fc4a4a96 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:11:53.157311 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-d275bed3-4d17-45ca-b988-a575fc4a4a96 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:11:53.157311 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:11:53.157311 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:11:53.157311 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:11:53.157311 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:11:53.157311 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:11:53.157311 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:11:53.157311 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:11:53.157311 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:11:53.157311 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:11:53.157311 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:11:53.157311 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:11:53.157311 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:11:53.157311 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:11:53.157311 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:11:53.157311 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:11:53.161021 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:11:53.161021 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:11:53.161021 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:11:53.161021 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:11:53.161021 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:11:53.161021 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:11:53.161021 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:11:53.161021 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:11:53.161021 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:11:53.161021 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:11:53.161021 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:11:53.161021 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 24 09:11:53.161021 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:11:53.161021 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:11:53.161021 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:11:53.161021 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:11:53.162072 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:11:53.162072 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:11:53.162072 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:11:53.162072 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:11:53.162072 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:11:53.162072 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:11:53.162072 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:11:53.162072 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:11:53.162072 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:11:53.162072 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:11:53.162072 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:11:53.162072 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 24 09:11:53.162072 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 630/1263] 10.222.0.16 () {56 vars in 1096 bytes} [Wed Aug 24 09:11:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:11:53.164532 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77c2293-e29a-47f7-ac9a-e0ee5fe35530 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:11:53.183172 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:11:53.260915 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77c2293-e29a-47f7-ac9a-e0ee5fe35530 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:11:53.295340 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a3bd34-25b7-4ea0-9113-426c23184330 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:11:53.295968 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a3bd34-25b7-4ea0-9113-426c23184330 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:11:53.296247 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a3bd34-25b7-4ea0-9113-426c23184330 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:11:53.296518 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a3bd34-25b7-4ea0-9113-426c23184330 None admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:11:53.297208 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a3bd34-25b7-4ea0-9113-426c23184330 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:11:53.297511 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a3bd34-25b7-4ea0-9113-426c23184330 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:11:53.298872 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a3bd34-25b7-4ea0-9113-426c23184330 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:11:53.307742 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 634/1264] 10.222.0.16 () {56 vars in 1099 bytes} [Wed Aug 24 09:11:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:11:53.313708 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d275bed3-4d17-45ca-b988-a575fc4a4a96 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:11:53.380474 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:11:53.413641 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d275bed3-4d17-45ca-b988-a575fc4a4a96 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:11:53.445439 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64777ed7-fbfd-4420-b798-0e5e2c4fcdc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:11:53.446060 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64777ed7-fbfd-4420-b798-0e5e2c4fcdc3 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:11:53.446336 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64777ed7-fbfd-4420-b798-0e5e2c4fcdc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:11:53.446607 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64777ed7-fbfd-4420-b798-0e5e2c4fcdc3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:11:53.457680 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64777ed7-fbfd-4420-b798-0e5e2c4fcdc3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:11:53.459139 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64777ed7-fbfd-4420-b798-0e5e2c4fcdc3 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:11:53.474903 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-64777ed7-fbfd-4420-b798-0e5e2c4fcdc3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:11:53.474903 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:11:53.474903 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:11:53.474903 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:11:53.474903 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:11:53.474903 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:11:53.474903 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:11:53.474903 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:11:53.474903 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:11:53.474903 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:11:53.474903 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:11:53.474903 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:11:53.474903 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:11:53.474903 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:11:53.474903 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:11:53.474903 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:11:53.476121 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:11:53.476121 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:11:53.476121 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:11:53.476121 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:11:53.476121 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:11:53.476121 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:11:53.476121 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:11:53.476121 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 24 09:11:53.476121 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:11:53.476121 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:11:53.476121 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:11:53.476121 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:11:53.476121 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:11:53.476121 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:11:53.476121 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:11:53.476121 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:11:53.476121 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:11:53.477270 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:11:53.477270 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:11:53.477270 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 24 09:11:53.477270 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 631/1265] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 24 09:11:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 09:11:53.482027 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a3bd34-25b7-4ea0-9113-426c23184330 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:11:53.561391 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:11:53.579867 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a3bd34-25b7-4ea0-9113-426c23184330 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:11:53.613890 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05b5be6-bbf1-42fc-a759-33348cb12435 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:11:53.614489 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05b5be6-bbf1-42fc-a759-33348cb12435 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:11:53.614770 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05b5be6-bbf1-42fc-a759-33348cb12435 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:11:53.615044 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05b5be6-bbf1-42fc-a759-33348cb12435 None admin] PATH_INFO: `/v3/domains` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:11:53.615710 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05b5be6-bbf1-42fc-a759-33348cb12435 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:11:53.616017 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05b5be6-bbf1-42fc-a759-33348cb12435 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:11:53.617421 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05b5be6-bbf1-42fc-a759-33348cb12435 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:11:53.628924 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 635/1266] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Aug 24 09:11:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:11:53.634883 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64777ed7-fbfd-4420-b798-0e5e2c4fcdc3 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:11:53.735120 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64777ed7-fbfd-4420-b798-0e5e2c4fcdc3 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:11:53.754255 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:11:53.767828 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a87da3e-53ae-427c-bc2d-df1eea39124f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:11:53.768455 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a87da3e-53ae-427c-bc2d-df1eea39124f None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:11:53.768732 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a87da3e-53ae-427c-bc2d-df1eea39124f None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:11:53.769003 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a87da3e-53ae-427c-bc2d-df1eea39124f None admin] PATH_INFO: `/v3/users/placement` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:11:53.776236 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a87da3e-53ae-427c-bc2d-df1eea39124f None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:11:53.777658 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a87da3e-53ae-427c-bc2d-df1eea39124f None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:11:53.789457 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-5a87da3e-53ae-427c-bc2d-df1eea39124f None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 09:11:53.789457 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:11:53.789457 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:11:53.789457 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:11:53.789457 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:11:53.789457 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:11:53.789457 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:11:53.789457 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:11:53.789457 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:11:53.789457 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:11:53.789457 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:11:53.789457 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:11:53.789457 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 09:11:53.789457 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 09:11:53.789457 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 09:11:53.789457 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 09:11:53.790542 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:11:53.790542 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:11:53.790542 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 09:11:53.790542 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:11:53.790542 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 09:11:53.790542 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:11:53.790542 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:11:53.790542 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:11:53.790542 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:11:53.790542 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:11:53.790542 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:11:53.790542 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 24 09:11:53.790542 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:11:53.790542 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:11:53.790542 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:11:53.790542 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:11:53.790542 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:11:53.790542 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:11:53.791486 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:11:53.791486 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:11:53.791486 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 09:11:53.791486 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 09:11:53.791486 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 09:11:53.791486 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 09:11:53.791486 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 09:11:53.791486 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 09:11:53.791486 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 09:11:53.791486 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 24 09:11:53.791486 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 632/1267] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 24 09:11:53 2022] GET /identity/v3/users/placement => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:11:53.796563 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05b5be6-bbf1-42fc-a759-33348cb12435 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:11:53.895633 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05b5be6-bbf1-42fc-a759-33348cb12435 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:11:53.929835 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98ca963-c6c7-4ec9-8d67-854c2a771832 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:11:53.930439 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98ca963-c6c7-4ec9-8d67-854c2a771832 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:11:53.930755 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98ca963-c6c7-4ec9-8d67-854c2a771832 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:11:53.931064 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98ca963-c6c7-4ec9-8d67-854c2a771832 None admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:11:53.931811 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98ca963-c6c7-4ec9-8d67-854c2a771832 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:11:53.932113 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98ca963-c6c7-4ec9-8d67-854c2a771832 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:11:53.933524 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98ca963-c6c7-4ec9-8d67-854c2a771832 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:11:53.942211 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:11:53.951385 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 636/1268] 10.222.0.16 () {56 vars in 1143 bytes} [Wed Aug 24 09:11:53 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:11:53.957697 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a87da3e-53ae-427c-bc2d-df1eea39124f None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:11:54.053935 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a87da3e-53ae-427c-bc2d-df1eea39124f None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:11:54.086400 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858f3dd5-20e6-4729-b7e5-7eb896e54dd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:11:54.087010 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858f3dd5-20e6-4729-b7e5-7eb896e54dd8 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:11:54.087287 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858f3dd5-20e6-4729-b7e5-7eb896e54dd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:11:54.087557 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858f3dd5-20e6-4729-b7e5-7eb896e54dd8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:11:54.098276 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858f3dd5-20e6-4729-b7e5-7eb896e54dd8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:11:54.099764 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858f3dd5-20e6-4729-b7e5-7eb896e54dd8 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:11:54.113144 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-858f3dd5-20e6-4729-b7e5-7eb896e54dd8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:11:54.113144 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:11:54.113144 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:11:54.113144 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:11:54.113144 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:11:54.113144 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:11:54.113144 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:11:54.113144 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:11:54.113144 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:11:54.113144 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:11:54.113144 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:11:54.113144 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:11:54.113144 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:11:54.113144 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:11:54.113144 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:11:54.113144 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:11:54.114261 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:11:54.114261 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:11:54.114261 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:11:54.114261 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:11:54.114261 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:11:54.114261 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:11:54.114261 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:11:54.114261 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 24 09:11:54.114261 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:11:54.114261 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:11:54.114261 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:11:54.114261 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:11:54.114261 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:11:54.114261 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:11:54.114261 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:11:54.114261 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:11:54.114261 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:11:54.115352 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:11:54.115352 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:11:54.115352 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 24 09:11:54.115352 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 633/1269] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 24 09:11:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:11:54.120044 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98ca963-c6c7-4ec9-8d67-854c2a771832 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:11:54.219650 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98ca963-c6c7-4ec9-8d67-854c2a771832 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:11:54.253749 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b715e00-9788-4101-8c89-0cff486cea8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:11:54.254352 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b715e00-9788-4101-8c89-0cff486cea8a None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:11:54.254635 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b715e00-9788-4101-8c89-0cff486cea8a None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:11:54.254904 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b715e00-9788-4101-8c89-0cff486cea8a None admin] PATH_INFO: `/v3/domains` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:11:54.255537 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b715e00-9788-4101-8c89-0cff486cea8a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:11:54.255891 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b715e00-9788-4101-8c89-0cff486cea8a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:11:54.257310 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b715e00-9788-4101-8c89-0cff486cea8a None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:11:54.268294 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 637/1270] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Aug 24 09:11:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:11:54.273977 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858f3dd5-20e6-4729-b7e5-7eb896e54dd8 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:11:54.307196 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:11:54.374911 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858f3dd5-20e6-4729-b7e5-7eb896e54dd8 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:11:54.409041 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a3a5bb-d202-481f-be7c-28752ce9bbf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:11:54.409671 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a3a5bb-d202-481f-be7c-28752ce9bbf3 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:11:54.409949 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a3a5bb-d202-481f-be7c-28752ce9bbf3 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:11:54.410222 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a3a5bb-d202-481f-be7c-28752ce9bbf3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:11:54.416775 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a3a5bb-d202-481f-be7c-28752ce9bbf3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:11:54.418202 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a3a5bb-d202-481f-be7c-28752ce9bbf3 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:11:54.430327 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-58a3a5bb-d202-481f-be7c-28752ce9bbf3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:11:54.430327 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:11:54.430327 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:11:54.430327 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:11:54.430327 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:11:54.430327 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:11:54.430327 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:11:54.430327 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:11:54.430327 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:11:54.430327 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:11:54.430327 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:11:54.430327 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:11:54.430327 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 09:11:54.430327 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 09:11:54.430327 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 09:11:54.431411 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 09:11:54.431411 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:11:54.431411 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:11:54.431411 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:11:54.431411 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:11:54.431411 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:11:54.431411 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:11:54.431411 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:11:54.431411 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 24 09:11:54.431411 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:11:54.431411 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:11:54.431411 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:11:54.431411 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:11:54.431411 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:11:54.431411 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:11:54.431411 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:11:54.431411 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:11:54.432701 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 09:11:54.432701 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 09:11:54.432701 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 09:11:54.432701 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 09:11:54.432701 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 09:11:54.432701 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 09:11:54.432701 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:11:54.432701 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 24 09:11:54.432701 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 634/1271] 10.222.0.16 () {56 vars in 1111 bytes} [Wed Aug 24 09:11:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:11:54.437254 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b715e00-9788-4101-8c89-0cff486cea8a None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:11:54.495929 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:11:54.536016 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b715e00-9788-4101-8c89-0cff486cea8a None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:11:54.568284 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d6fa98-ecc3-4ca9-a7a9-6a632ea4bbca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:11:54.568878 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d6fa98-ecc3-4ca9-a7a9-6a632ea4bbca None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:11:54.569154 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d6fa98-ecc3-4ca9-a7a9-6a632ea4bbca None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:11:54.569421 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d6fa98-ecc3-4ca9-a7a9-6a632ea4bbca None admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:11:54.570100 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d6fa98-ecc3-4ca9-a7a9-6a632ea4bbca None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:11:54.570399 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d6fa98-ecc3-4ca9-a7a9-6a632ea4bbca None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:11:54.571841 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d6fa98-ecc3-4ca9-a7a9-6a632ea4bbca None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:11:54.583786 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 638/1272] 10.222.0.16 () {56 vars in 1148 bytes} [Wed Aug 24 09:11:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:11:54.589867 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a3a5bb-d202-481f-be7c-28752ce9bbf3 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:11:54.680260 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:11:54.688847 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a3a5bb-d202-481f-be7c-28752ce9bbf3 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:11:54.725960 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e43972e-11c9-4bb5-bdc4-417c1b304793 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:11:54.726609 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e43972e-11c9-4bb5-bdc4-417c1b304793 None admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:11:54.726888 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e43972e-11c9-4bb5-bdc4-417c1b304793 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:11:54.727163 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e43972e-11c9-4bb5-bdc4-417c1b304793 None admin] PATH_INFO: `/v3/projects/c467e88e2fb0491bacefcf6d2ce326d9/users/80cada22039d4f11935412ee578acd05/roles/15f634d4a2d247ddab90c458e5e1c141` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:11:54.729142 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e43972e-11c9-4bb5-bdc4-417c1b304793 None admin] RBAC: Authorizing `identity:create_grant(project_id=c467e88e2fb0491bacefcf6d2ce326d9, user_id=80cada22039d4f11935412ee578acd05, role_id=15f634d4a2d247ddab90c458e5e1c141)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:11:54.730594 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e43972e-11c9-4bb5-bdc4-417c1b304793 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:11:54.892364 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 635/1273] 10.222.0.16 () {58 vars in 1439 bytes} [Wed Aug 24 09:11:54 2022] PUT /identity/v3/projects/c467e88e2fb0491bacefcf6d2ce326d9/users/80cada22039d4f11935412ee578acd05/roles/15f634d4a2d247ddab90c458e5e1c141 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:11:56.754019 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6717e1e8-53b0-4ca4-9deb-93306f235525 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:11:56.754530 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6717e1e8-53b0-4ca4-9deb-93306f235525 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:11:56.754530 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6717e1e8-53b0-4ca4-9deb-93306f235525 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:11:56.756102 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 639/1274] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:11:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:11:56.761542 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83bd30e-4e6a-4d98-aaac-3d97ed874970 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:11:56.761938 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83bd30e-4e6a-4d98-aaac-3d97ed874970 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:11:56.762085 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83bd30e-4e6a-4d98-aaac-3d97ed874970 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:11:56.780556 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-f83bd30e-4e6a-4d98-aaac-3d97ed874970 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:11:56.781726 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-f83bd30e-4e6a-4d98-aaac-3d97ed874970 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:11:56.848743 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 636/1275] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:11:56 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 09:11:57.120089 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6717e1e8-53b0-4ca4-9deb-93306f235525 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:11:57.220703 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6717e1e8-53b0-4ca4-9deb-93306f235525 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:11:57.238173 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:11:57.255839 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cf700e-2063-4617-8a3d-b16507c8395e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:11:57.256465 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cf700e-2063-4617-8a3d-b16507c8395e None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:11:57.256743 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cf700e-2063-4617-8a3d-b16507c8395e None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:11:57.257015 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cf700e-2063-4617-8a3d-b16507c8395e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:11:57.266813 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cf700e-2063-4617-8a3d-b16507c8395e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:11:57.271479 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-19cf700e-2063-4617-8a3d-b16507c8395e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:11:57.271479 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:11:57.271479 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:11:57.271479 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:11:57.271479 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:11:57.271479 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:11:57.271479 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:11:57.271479 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:11:57.271479 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:11:57.271479 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:11:57.271479 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:11:57.271479 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:11:57.271479 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:11:57.271479 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:11:57.271479 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:11:57.271479 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:11:57.272626 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:11:57.272626 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:11:57.272626 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:11:57.272626 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:11:57.272626 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:11:57.272626 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:11:57.272626 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:11:57.272626 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:11:57.272626 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:11:57.272626 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:11:57.272626 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:11:57.272626 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Aug 24 09:11:57.272626 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:11:57.272626 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:11:57.272626 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:11:57.272626 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:11:57.273530 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:11:57.273530 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:11:57.273530 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:11:57.273530 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:11:57.273530 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:11:57.273530 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:11:57.273530 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:11:57.273530 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:11:57.273530 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:11:57.273530 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:11:57.273530 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:11:57.273530 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Aug 24 09:11:57.273530 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 640/1276] 10.222.0.16 () {56 vars in 1096 bytes} [Wed Aug 24 09:11:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:11:57.278678 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83bd30e-4e6a-4d98-aaac-3d97ed874970 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:11:57.358715 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:11:57.376462 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83bd30e-4e6a-4d98-aaac-3d97ed874970 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:11:57.409185 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abd0a4e-31f7-4939-9470-fbe26591b95d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:11:57.409793 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abd0a4e-31f7-4939-9470-fbe26591b95d None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:11:57.410121 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abd0a4e-31f7-4939-9470-fbe26591b95d None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:11:57.410394 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abd0a4e-31f7-4939-9470-fbe26591b95d None admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:11:57.411104 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abd0a4e-31f7-4939-9470-fbe26591b95d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:11:57.411504 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abd0a4e-31f7-4939-9470-fbe26591b95d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:11:57.412880 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abd0a4e-31f7-4939-9470-fbe26591b95d None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:11:57.421093 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 637/1277] 10.222.0.16 () {56 vars in 1099 bytes} [Wed Aug 24 09:11:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:11:57.427394 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cf700e-2063-4617-8a3d-b16507c8395e None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:11:57.526428 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cf700e-2063-4617-8a3d-b16507c8395e None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:11:57.561066 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97401c6-5c06-4c08-897e-55e95a9e58d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:11:57.561686 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97401c6-5c06-4c08-897e-55e95a9e58d1 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:11:57.561983 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97401c6-5c06-4c08-897e-55e95a9e58d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:11:57.562249 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97401c6-5c06-4c08-897e-55e95a9e58d1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:11:57.572302 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97401c6-5c06-4c08-897e-55e95a9e58d1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:11:57.573812 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97401c6-5c06-4c08-897e-55e95a9e58d1 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:11:57.588080 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-d97401c6-5c06-4c08-897e-55e95a9e58d1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:11:57.588080 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:11:57.588080 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:11:57.588080 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:11:57.588080 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:11:57.588080 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:11:57.588080 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:11:57.588080 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:11:57.588080 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:11:57.588080 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:11:57.588080 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:11:57.588080 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:11:57.588080 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:11:57.588080 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:11:57.588080 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:11:57.588080 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:11:57.593915 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:11:57.593915 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:11:57.593915 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:11:57.593915 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:11:57.593915 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:11:57.593915 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:11:57.593915 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:11:57.593915 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Aug 24 09:11:57.593915 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:11:57.593915 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:11:57.593915 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:11:57.593915 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:11:57.593915 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:11:57.593915 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:11:57.593915 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:11:57.593915 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:11:57.593915 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:11:57.595759 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:11:57.595759 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:11:57.595759 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Aug 24 09:11:57.595759 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 641/1278] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 24 09:11:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:11:57.596049 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abd0a4e-31f7-4939-9470-fbe26591b95d None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:11:57.692013 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abd0a4e-31f7-4939-9470-fbe26591b95d None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:11:57.725346 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aeebc2f-4fc1-4dc0-a6a7-60fa8235c709 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:11:57.725938 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aeebc2f-4fc1-4dc0-a6a7-60fa8235c709 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:11:57.726212 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aeebc2f-4fc1-4dc0-a6a7-60fa8235c709 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:11:57.726479 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aeebc2f-4fc1-4dc0-a6a7-60fa8235c709 None admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:11:57.727112 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aeebc2f-4fc1-4dc0-a6a7-60fa8235c709 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:11:57.727413 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aeebc2f-4fc1-4dc0-a6a7-60fa8235c709 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:11:57.728786 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aeebc2f-4fc1-4dc0-a6a7-60fa8235c709 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:11:57.739997 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 638/1279] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Aug 24 09:11:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:11:57.745625 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97401c6-5c06-4c08-897e-55e95a9e58d1 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:11:57.777234 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:11:57.849637 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97401c6-5c06-4c08-897e-55e95a9e58d1 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:11:57.882086 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a47247f-8c52-45cb-9d20-b33e2e80b32e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:11:57.882707 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a47247f-8c52-45cb-9d20-b33e2e80b32e None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:11:57.882990 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a47247f-8c52-45cb-9d20-b33e2e80b32e None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:11:57.883263 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a47247f-8c52-45cb-9d20-b33e2e80b32e None admin] PATH_INFO: `/v3/users/placement` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:11:57.890516 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a47247f-8c52-45cb-9d20-b33e2e80b32e None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:11:57.892473 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a47247f-8c52-45cb-9d20-b33e2e80b32e None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:11:57.903352 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-4a47247f-8c52-45cb-9d20-b33e2e80b32e None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 09:11:57.903352 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:11:57.903352 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:11:57.903352 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:11:57.903352 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:11:57.903352 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:11:57.903352 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:11:57.903352 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:11:57.903352 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:11:57.903352 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:11:57.903352 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:11:57.903352 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:11:57.903352 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 09:11:57.903352 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 09:11:57.903352 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 09:11:57.903352 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 09:11:57.904560 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:11:57.904560 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:11:57.904560 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 09:11:57.904560 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:11:57.904560 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 09:11:57.904560 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:11:57.904560 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:11:57.904560 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:11:57.904560 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:11:57.904560 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:11:57.904560 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:11:57.904560 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Aug 24 09:11:57.904560 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:11:57.904560 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:11:57.904560 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:11:57.904560 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:11:57.904560 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:11:57.904560 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:11:57.905810 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:11:57.905810 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:11:57.905810 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 09:11:57.905810 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 09:11:57.905810 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 09:11:57.905810 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 09:11:57.905810 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 09:11:57.905810 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 09:11:57.905810 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 09:11:57.905810 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Aug 24 09:11:57.905810 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 642/1280] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 24 09:11:57 2022] GET /identity/v3/users/placement => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:11:57.910918 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aeebc2f-4fc1-4dc0-a6a7-60fa8235c709 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:11:57.914772 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:11:58.014518 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aeebc2f-4fc1-4dc0-a6a7-60fa8235c709 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:11:58.048752 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44de3a4d-e56d-4809-b5b3-dec5a0aee42b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:11:58.049358 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44de3a4d-e56d-4809-b5b3-dec5a0aee42b None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:11:58.049633 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44de3a4d-e56d-4809-b5b3-dec5a0aee42b None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:11:58.049903 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44de3a4d-e56d-4809-b5b3-dec5a0aee42b None admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:11:58.050670 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44de3a4d-e56d-4809-b5b3-dec5a0aee42b None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:11:58.050971 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44de3a4d-e56d-4809-b5b3-dec5a0aee42b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:11:58.052409 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44de3a4d-e56d-4809-b5b3-dec5a0aee42b None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:11:58.070179 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 639/1281] 10.222.0.16 () {56 vars in 1143 bytes} [Wed Aug 24 09:11:57 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:11:58.076031 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a47247f-8c52-45cb-9d20-b33e2e80b32e None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:11:58.167802 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:11:58.177163 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a47247f-8c52-45cb-9d20-b33e2e80b32e None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:11:58.210492 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26480633-eaf1-499c-a51a-2786c32ecaf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:11:58.211112 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26480633-eaf1-499c-a51a-2786c32ecaf0 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:11:58.211378 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26480633-eaf1-499c-a51a-2786c32ecaf0 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:11:58.211677 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26480633-eaf1-499c-a51a-2786c32ecaf0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:11:58.278388 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26480633-eaf1-499c-a51a-2786c32ecaf0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:11:58.279908 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26480633-eaf1-499c-a51a-2786c32ecaf0 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:11:58.369318 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:11:58.380073 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 643/1282] 10.222.0.16 () {58 vars in 1282 bytes} [Wed Aug 24 09:11:58 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 09:11:58.386185 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44de3a4d-e56d-4809-b5b3-dec5a0aee42b None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:11:58.448344 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:11:58.485972 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44de3a4d-e56d-4809-b5b3-dec5a0aee42b None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:11:58.521040 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c60b3e-8af1-45bc-afdc-3ebaa86e1717 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:11:58.521664 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c60b3e-8af1-45bc-afdc-3ebaa86e1717 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:11:58.521941 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c60b3e-8af1-45bc-afdc-3ebaa86e1717 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:11:58.522211 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c60b3e-8af1-45bc-afdc-3ebaa86e1717 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:11:58.537972 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c60b3e-8af1-45bc-afdc-3ebaa86e1717 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:11:58.537972 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c60b3e-8af1-45bc-afdc-3ebaa86e1717 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:11:58.549153 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-f6c60b3e-8af1-45bc-afdc-3ebaa86e1717 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:11:58.549153 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:11:58.549153 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:11:58.549153 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:11:58.549153 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:11:58.549153 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:11:58.549153 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:11:58.549153 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:11:58.549153 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:11:58.549153 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:11:58.549153 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:11:58.549153 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:11:58.549153 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:11:58.549153 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:11:58.549153 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:11:58.549153 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:11:58.550265 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:11:58.550265 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:11:58.550265 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:11:58.550265 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:11:58.550265 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:11:58.550265 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:11:58.550265 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:11:58.550265 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 24 09:11:58.550265 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:11:58.550265 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:11:58.550265 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:11:58.550265 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:11:58.550265 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:11:58.550265 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:11:58.550265 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:11:58.550265 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:11:58.550265 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:11:58.551410 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:11:58.551410 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:11:58.551410 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 24 09:11:58.551410 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 640/1283] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 24 09:11:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:11:58.556274 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26480633-eaf1-499c-a51a-2786c32ecaf0 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:11:58.656000 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26480633-eaf1-499c-a51a-2786c32ecaf0 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:11:58.690010 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909a9537-e176-43d5-8e6b-9ae0df806d41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:11:58.690647 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909a9537-e176-43d5-8e6b-9ae0df806d41 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:11:58.690929 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909a9537-e176-43d5-8e6b-9ae0df806d41 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:11:58.691203 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909a9537-e176-43d5-8e6b-9ae0df806d41 None admin] PATH_INFO: `/v3/domains` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:11:58.691876 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909a9537-e176-43d5-8e6b-9ae0df806d41 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:11:58.692183 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909a9537-e176-43d5-8e6b-9ae0df806d41 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:11:58.693551 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909a9537-e176-43d5-8e6b-9ae0df806d41 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:11:58.706879 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 644/1284] 10.222.0.16 () {56 vars in 1109 bytes} [Wed Aug 24 09:11:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:11:58.712628 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c60b3e-8af1-45bc-afdc-3ebaa86e1717 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:11:58.810585 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c60b3e-8af1-45bc-afdc-3ebaa86e1717 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:11:58.824604 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:11:58.844995 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a315bd08-eb93-48dd-b2da-64858b8dad8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:11:58.845622 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a315bd08-eb93-48dd-b2da-64858b8dad8e None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:11:58.845903 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a315bd08-eb93-48dd-b2da-64858b8dad8e None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:11:58.846175 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a315bd08-eb93-48dd-b2da-64858b8dad8e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:11:58.852679 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a315bd08-eb93-48dd-b2da-64858b8dad8e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:11:58.854158 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a315bd08-eb93-48dd-b2da-64858b8dad8e None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:11:58.865070 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-a315bd08-eb93-48dd-b2da-64858b8dad8e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:11:58.865070 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:11:58.865070 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:11:58.865070 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:11:58.865070 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:11:58.865070 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:11:58.865070 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:11:58.865070 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:11:58.865070 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:11:58.865070 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:11:58.865070 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:11:58.865070 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:11:58.865070 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 09:11:58.865070 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 09:11:58.865070 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 09:11:58.866141 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 09:11:58.866141 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:11:58.866141 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:11:58.866141 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:11:58.866141 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:11:58.866141 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:11:58.866141 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:11:58.866141 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:11:58.866141 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 24 09:11:58.866141 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:11:58.866141 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:11:58.866141 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:11:58.866141 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:11:58.866141 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:11:58.866141 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:11:58.866141 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:11:58.866141 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:11:58.867397 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 09:11:58.867397 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 09:11:58.867397 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 09:11:58.867397 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 09:11:58.867397 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 09:11:58.867397 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 09:11:58.867397 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:11:58.867397 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 24 09:11:58.867397 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 641/1285] 10.222.0.16 () {56 vars in 1111 bytes} [Wed Aug 24 09:11:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:11:58.872170 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909a9537-e176-43d5-8e6b-9ae0df806d41 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:11:58.922820 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:11:58.973464 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909a9537-e176-43d5-8e6b-9ae0df806d41 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:11:59.008036 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8544de0c-7f80-4948-9c10-6793c1e0276f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:11:59.008656 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8544de0c-7f80-4948-9c10-6793c1e0276f None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:11:59.008955 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8544de0c-7f80-4948-9c10-6793c1e0276f None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:11:59.009437 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8544de0c-7f80-4948-9c10-6793c1e0276f None admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:11:59.010690 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8544de0c-7f80-4948-9c10-6793c1e0276f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:11:59.011251 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8544de0c-7f80-4948-9c10-6793c1e0276f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:11:59.014123 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8544de0c-7f80-4948-9c10-6793c1e0276f None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:11:59.032327 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 645/1286] 10.222.0.16 () {56 vars in 1148 bytes} [Wed Aug 24 09:11:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:11:59.038450 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a315bd08-eb93-48dd-b2da-64858b8dad8e None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:11:59.136509 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a315bd08-eb93-48dd-b2da-64858b8dad8e None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:11:59.169009 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9f00d5-1817-461c-948d-8e8837b6d5f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:11:59.169609 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9f00d5-1817-461c-948d-8e8837b6d5f0 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:11:59.169888 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9f00d5-1817-461c-948d-8e8837b6d5f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:11:59.170162 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9f00d5-1817-461c-948d-8e8837b6d5f0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:11:59.170883 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9f00d5-1817-461c-948d-8e8837b6d5f0 None admin] RBAC: Adding query filter params (role.id=15f634d4a2d247ddab90c458e5e1c141, scope.project.id=c467e88e2fb0491bacefcf6d2ce326d9, user.id=80cada22039d4f11935412ee578acd05) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:11:59.171194 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9f00d5-1817-461c-948d-8e8837b6d5f0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:11:59.173188 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9f00d5-1817-461c-948d-8e8837b6d5f0 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:11:59.178830 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 642/1287] 10.222.0.16 () {56 vars in 1374 bytes} [Wed Aug 24 09:11:59 2022] GET /identity/v3/role_assignments?user.id=80cada22039d4f11935412ee578acd05&scope.project.id=c467e88e2fb0491bacefcf6d2ce326d9&role.id=15f634d4a2d247ddab90c458e5e1c141 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:12:01.125997 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f781c31b-4fb8-4ca4-a040-ad1c91c7602f None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:12:01.126533 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f781c31b-4fb8-4ca4-a040-ad1c91c7602f None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:12:01.126533 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f781c31b-4fb8-4ca4-a040-ad1c91c7602f None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:12:01.127904 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 646/1288] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:12:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:12:01.133113 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63e9a09-c583-46be-9fc9-cfac527837b3 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:12:01.133549 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63e9a09-c583-46be-9fc9-cfac527837b3 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:12:01.133668 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63e9a09-c583-46be-9fc9-cfac527837b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:12:01.151386 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-c63e9a09-c583-46be-9fc9-cfac527837b3 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:12:01.152600 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-c63e9a09-c583-46be-9fc9-cfac527837b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:12:01.182084 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:12:01.217465 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 643/1289] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:12:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 09:12:01.482834 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f781c31b-4fb8-4ca4-a040-ad1c91c7602f None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:12:01.581821 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f781c31b-4fb8-4ca4-a040-ad1c91c7602f None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:12:01.592374 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:12:01.614997 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ef51b7-8218-4a67-ab4b-df7f14300a3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:12:01.615624 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ef51b7-8218-4a67-ab4b-df7f14300a3e None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:12:01.615903 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ef51b7-8218-4a67-ab4b-df7f14300a3e None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:12:01.616169 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ef51b7-8218-4a67-ab4b-df7f14300a3e None admin] PATH_INFO: `/v3/services/placement` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:12:01.616828 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ef51b7-8218-4a67-ab4b-df7f14300a3e None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:12:01.618186 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ef51b7-8218-4a67-ab4b-df7f14300a3e None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:12:01.627300 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-92ef51b7-8218-4a67-ab4b-df7f14300a3e None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 09:12:01.627300 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:12:01.627300 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 09:12:01.627300 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 09:12:01.627300 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 09:12:01.627300 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 09:12:01.627300 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 09:12:01.627300 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:12:01.627300 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 09:12:01.627300 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Aug 24 09:12:01.627300 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 09:12:01.627300 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Aug 24 09:12:01.627300 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:12:01.627300 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:12:01.627300 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:12:01.627300 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:12:01.627300 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:12:01.628567 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:12:01.628567 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:12:01.628567 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:12:01.628567 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:12:01.628567 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:12:01.628567 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:12:01.628567 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 09:12:01.628567 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 09:12:01.628567 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 09:12:01.628567 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 09:12:01.628567 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:12:01.628567 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:12:01.628567 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:12:01.628567 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:12:01.628567 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:12:01.628567 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:12:01.629445 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:12:01.629445 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Aug 24 09:12:01.629445 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:12:01.629445 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:12:01.629445 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:12:01.629445 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:12:01.629445 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:12:01.629445 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:12:01.629445 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:12:01.629445 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:12:01.629445 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 09:12:01.629445 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:12:01.629445 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 09:12:01.629445 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Aug 24 09:12:01.629445 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 647/1290] 10.222.0.16 () {56 vars in 1117 bytes} [Wed Aug 24 09:12:01 2022] GET /identity/v3/services/placement => generated 90 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:12:01.634390 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63e9a09-c583-46be-9fc9-cfac527837b3 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:12:01.729445 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63e9a09-c583-46be-9fc9-cfac527837b3 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:12:01.763661 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8685a60f-6840-4902-9e55-d9f497e52814 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:12:01.764198 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8685a60f-6840-4902-9e55-d9f497e52814 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:12:01.764479 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8685a60f-6840-4902-9e55-d9f497e52814 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:12:01.764750 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8685a60f-6840-4902-9e55-d9f497e52814 None admin] PATH_INFO: `/v3/services` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:12:01.765377 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8685a60f-6840-4902-9e55-d9f497e52814 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:12:01.765677 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8685a60f-6840-4902-9e55-d9f497e52814 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:12:01.767072 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8685a60f-6840-4902-9e55-d9f497e52814 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:12:01.771761 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 644/1291] 10.222.0.16 () {56 vars in 1116 bytes} [Wed Aug 24 09:12:01 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:12:01.777834 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ef51b7-8218-4a67-ab4b-df7f14300a3e None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:12:01.876452 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ef51b7-8218-4a67-ab4b-df7f14300a3e None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:12:01.908674 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80862832-8fc3-4f86-a734-c5016f9d0960 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:12:01.909255 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80862832-8fc3-4f86-a734-c5016f9d0960 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:12:01.909535 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80862832-8fc3-4f86-a734-c5016f9d0960 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:12:01.909805 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80862832-8fc3-4f86-a734-c5016f9d0960 None admin] PATH_INFO: `/v3/services` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:12:01.910420 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80862832-8fc3-4f86-a734-c5016f9d0960 None admin] RBAC: Adding query filter params (type=placement) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:12:01.910722 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80862832-8fc3-4f86-a734-c5016f9d0960 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:12:01.912107 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80862832-8fc3-4f86-a734-c5016f9d0960 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:12:01.916648 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 648/1292] 10.222.0.16 () {56 vars in 1116 bytes} [Wed Aug 24 09:12:01 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:12:03.824333 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d13f933-99d8-44a5-8193-7cd258b1e8cf None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:12:03.824897 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d13f933-99d8-44a5-8193-7cd258b1e8cf None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:12:03.825011 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d13f933-99d8-44a5-8193-7cd258b1e8cf None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:12:03.826594 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 645/1293] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:12:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:12:03.831711 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa874985-518f-4552-93ac-c75c42f3702d None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:12:03.832175 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa874985-518f-4552-93ac-c75c42f3702d None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:12:03.832267 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa874985-518f-4552-93ac-c75c42f3702d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:12:03.849934 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-aa874985-518f-4552-93ac-c75c42f3702d None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:12:03.851146 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-aa874985-518f-4552-93ac-c75c42f3702d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:12:03.880374 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:12:03.915999 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 649/1294] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:12:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 09:12:04.186883 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d13f933-99d8-44a5-8193-7cd258b1e8cf None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:12:04.217852 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:12:04.287814 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d13f933-99d8-44a5-8193-7cd258b1e8cf None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:12:04.321338 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a463c6c-809c-4bc4-bf84-34d6fb129845 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:12:04.321937 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a463c6c-809c-4bc4-bf84-34d6fb129845 None admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:12:04.322210 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a463c6c-809c-4bc4-bf84-34d6fb129845 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:12:04.322475 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a463c6c-809c-4bc4-bf84-34d6fb129845 None admin] PATH_INFO: `/v3/services` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:12:04.323113 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a463c6c-809c-4bc4-bf84-34d6fb129845 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:12:04.324501 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a463c6c-809c-4bc4-bf84-34d6fb129845 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:12:04.469232 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 646/1295] 10.222.0.16 () {60 vars in 1141 bytes} [Wed Aug 24 09:12:04 2022] POST /identity/v3/services => generated 246 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:12:06.389357 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380e54d7-c47f-4fbd-a435-d829c5c6839b None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:12:06.389902 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380e54d7-c47f-4fbd-a435-d829c5c6839b None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:12:06.389902 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380e54d7-c47f-4fbd-a435-d829c5c6839b None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:12:06.391900 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 650/1296] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:12:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:12:06.397544 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff80fd3-05e4-43ac-8b4c-0c04e96e2bb1 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:12:06.398028 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff80fd3-05e4-43ac-8b4c-0c04e96e2bb1 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:12:06.398122 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff80fd3-05e4-43ac-8b4c-0c04e96e2bb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:12:06.415781 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-1ff80fd3-05e4-43ac-8b4c-0c04e96e2bb1 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:12:06.419753 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-1ff80fd3-05e4-43ac-8b4c-0c04e96e2bb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:12:06.488193 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 647/1297] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:12:06 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 09:12:06.755817 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380e54d7-c47f-4fbd-a435-d829c5c6839b None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:12:06.855012 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380e54d7-c47f-4fbd-a435-d829c5c6839b None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:12:06.888660 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f92ec9-582e-422e-afdf-2090aa4ecf47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:12:06.889277 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f92ec9-582e-422e-afdf-2090aa4ecf47 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:12:06.889550 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f92ec9-582e-422e-afdf-2090aa4ecf47 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:12:06.889819 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f92ec9-582e-422e-afdf-2090aa4ecf47 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:12:06.890389 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f92ec9-582e-422e-afdf-2090aa4ecf47 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:12:06.891800 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f92ec9-582e-422e-afdf-2090aa4ecf47 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:12:06.900187 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-f1f92ec9-582e-422e-afdf-2090aa4ecf47 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 09:12:06.900187 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:12:06.900187 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 09:12:06.900187 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 09:12:06.900187 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 09:12:06.900187 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 09:12:06.900187 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 09:12:06.900187 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:12:06.900187 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 09:12:06.900187 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Aug 24 09:12:06.900187 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 09:12:06.900187 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Aug 24 09:12:06.900187 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:12:06.900187 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:12:06.900187 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:12:06.900187 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:12:06.900187 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:12:06.902679 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:12:06.902679 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:12:06.902679 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:12:06.902679 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:12:06.902679 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:12:06.902679 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:12:06.902679 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 09:12:06.902679 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 09:12:06.902679 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 09:12:06.902679 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 09:12:06.902679 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:12:06.902679 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:12:06.902679 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:12:06.902679 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:12:06.902679 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:12:06.902679 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:12:06.903777 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:12:06.903777 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Aug 24 09:12:06.903777 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:12:06.903777 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:12:06.903777 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:12:06.903777 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:12:06.903777 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:12:06.903777 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:12:06.903777 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:12:06.903777 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:12:06.903777 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 09:12:06.903777 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:12:06.903777 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 09:12:06.903777 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Aug 24 09:12:06.903777 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 651/1298] 10.222.0.16 () {56 vars in 1117 bytes} [Wed Aug 24 09:12:06 2022] GET /identity/v3/services/placement => generated 90 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:12:06.907356 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff80fd3-05e4-43ac-8b4c-0c04e96e2bb1 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:12:06.926448 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:12:07.003338 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff80fd3-05e4-43ac-8b4c-0c04e96e2bb1 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:12:07.035218 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0118db-dd86-461e-b22b-e107d9743358 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:12:07.035831 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0118db-dd86-461e-b22b-e107d9743358 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:12:07.036107 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0118db-dd86-461e-b22b-e107d9743358 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:12:07.036377 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0118db-dd86-461e-b22b-e107d9743358 None admin] PATH_INFO: `/v3/services` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:12:07.037002 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0118db-dd86-461e-b22b-e107d9743358 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:12:07.037303 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0118db-dd86-461e-b22b-e107d9743358 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:12:07.038647 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0118db-dd86-461e-b22b-e107d9743358 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:12:07.043460 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 648/1299] 10.222.0.16 () {56 vars in 1116 bytes} [Wed Aug 24 09:12:06 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:12:07.050267 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f92ec9-582e-422e-afdf-2090aa4ecf47 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:12:07.081233 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:12:07.151668 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f92ec9-582e-422e-afdf-2090aa4ecf47 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:12:07.184779 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08fb080-2ad2-433f-99f3-2044e3d59e4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:12:07.185375 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08fb080-2ad2-433f-99f3-2044e3d59e4a None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:12:07.185655 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08fb080-2ad2-433f-99f3-2044e3d59e4a None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:12:07.185928 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08fb080-2ad2-433f-99f3-2044e3d59e4a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:12:07.186619 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08fb080-2ad2-433f-99f3-2044e3d59e4a None admin] RBAC: Adding query filter params (interface=public, service_id=17cc4408ab894549870f5b6ef619d800, region_id=RegionOne) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:12:07.186923 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08fb080-2ad2-433f-99f3-2044e3d59e4a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:12:07.188309 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08fb080-2ad2-433f-99f3-2044e3d59e4a None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:12:07.193131 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 652/1300] 10.222.0.16 () {56 vars in 1251 bytes} [Wed Aug 24 09:12:07 2022] GET /identity/v3/endpoints?service_id=17cc4408ab894549870f5b6ef619d800&interface=public®ion_id=RegionOne => generated 194 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:12:07.198979 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0118db-dd86-461e-b22b-e107d9743358 None admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:12:07.279218 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:12:07.297100 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0118db-dd86-461e-b22b-e107d9743358 None admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:12:07.330946 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f61c26-c2e8-4914-973b-8ca10801ebc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:12:07.331562 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f61c26-c2e8-4914-973b-8ca10801ebc8 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:12:07.331854 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f61c26-c2e8-4914-973b-8ca10801ebc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:12:07.332125 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f61c26-c2e8-4914-973b-8ca10801ebc8 None admin] PATH_INFO: `/v3/services` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:12:07.332721 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f61c26-c2e8-4914-973b-8ca10801ebc8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:12:07.334072 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f61c26-c2e8-4914-973b-8ca10801ebc8 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:12:07.341195 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 649/1301] 10.222.0.16 () {56 vars in 1087 bytes} [Wed Aug 24 09:12:07 2022] GET /identity/v3/services => generated 2191 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:12:09.231703 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2179f0a0-54c9-4cc0-8de4-19b931bfd4f9 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:12:09.232260 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2179f0a0-54c9-4cc0-8de4-19b931bfd4f9 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:12:09.232260 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2179f0a0-54c9-4cc0-8de4-19b931bfd4f9 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:12:09.233594 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 653/1302] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:12:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:12:09.238924 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc90cc9b-6a54-49af-bc6c-8e584090ee26 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:12:09.239349 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc90cc9b-6a54-49af-bc6c-8e584090ee26 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:12:09.239478 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc90cc9b-6a54-49af-bc6c-8e584090ee26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:12:09.257646 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-cc90cc9b-6a54-49af-bc6c-8e584090ee26 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:12:09.258828 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-cc90cc9b-6a54-49af-bc6c-8e584090ee26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:12:09.324415 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 650/1303] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:12:09 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 09:12:09.592029 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2179f0a0-54c9-4cc0-8de4-19b931bfd4f9 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:12:09.687233 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:12:09.694250 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2179f0a0-54c9-4cc0-8de4-19b931bfd4f9 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:12:09.726193 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef771c6-bee9-4c02-b77c-db65a2214ea1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:12:09.726807 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef771c6-bee9-4c02-b77c-db65a2214ea1 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:12:09.727083 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef771c6-bee9-4c02-b77c-db65a2214ea1 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:12:09.727351 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef771c6-bee9-4c02-b77c-db65a2214ea1 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:12:09.727949 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef771c6-bee9-4c02-b77c-db65a2214ea1 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:12:09.729305 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef771c6-bee9-4c02-b77c-db65a2214ea1 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:12:09.738199 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-5ef771c6-bee9-4c02-b77c-db65a2214ea1 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 09:12:09.738199 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:12:09.738199 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 09:12:09.738199 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 09:12:09.738199 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 09:12:09.738199 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 09:12:09.738199 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 09:12:09.738199 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:12:09.738199 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 09:12:09.738199 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Aug 24 09:12:09.738199 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 09:12:09.738199 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Aug 24 09:12:09.738199 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:12:09.738199 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:12:09.738199 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:12:09.738199 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:12:09.738199 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:12:09.749270 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:12:09.749270 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:12:09.749270 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:12:09.749270 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:12:09.749270 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:12:09.749270 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:12:09.749270 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 09:12:09.749270 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 09:12:09.749270 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 09:12:09.749270 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 09:12:09.749270 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:12:09.749270 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:12:09.749270 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:12:09.749270 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:12:09.749270 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:12:09.749270 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:12:09.751020 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:12:09.751020 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Aug 24 09:12:09.751020 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:12:09.751020 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:12:09.751020 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:12:09.751020 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:12:09.751020 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:12:09.751020 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:12:09.751020 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:12:09.751020 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:12:09.751020 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 09:12:09.751020 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:12:09.751020 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 09:12:09.751020 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Aug 24 09:12:09.751020 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 654/1304] 10.222.0.16 () {56 vars in 1117 bytes} [Wed Aug 24 09:12:09 2022] GET /identity/v3/services/placement => generated 90 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:12:09.752099 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc90cc9b-6a54-49af-bc6c-8e584090ee26 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:12:09.821077 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:12:09.843008 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc90cc9b-6a54-49af-bc6c-8e584090ee26 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:12:09.878348 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7345219f-2643-4d17-87c2-fa2b8d8201a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:12:09.878961 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7345219f-2643-4d17-87c2-fa2b8d8201a1 None admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:12:09.879240 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7345219f-2643-4d17-87c2-fa2b8d8201a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:12:09.879509 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7345219f-2643-4d17-87c2-fa2b8d8201a1 None admin] PATH_INFO: `/v3/services` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:12:09.880183 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7345219f-2643-4d17-87c2-fa2b8d8201a1 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:12:09.880475 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7345219f-2643-4d17-87c2-fa2b8d8201a1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:12:09.881827 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7345219f-2643-4d17-87c2-fa2b8d8201a1 None admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:12:09.886807 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 651/1305] 10.222.0.16 () {56 vars in 1116 bytes} [Wed Aug 24 09:12:09 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:12:09.892664 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef771c6-bee9-4c02-b77c-db65a2214ea1 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:12:09.990520 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef771c6-bee9-4c02-b77c-db65a2214ea1 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:12:10.023281 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46426016-f3d0-4b16-a83c-11c2f2e8fbd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:12:10.023894 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46426016-f3d0-4b16-a83c-11c2f2e8fbd2 None admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:12:10.024170 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46426016-f3d0-4b16-a83c-11c2f2e8fbd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:12:10.024440 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46426016-f3d0-4b16-a83c-11c2f2e8fbd2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:12:10.025093 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46426016-f3d0-4b16-a83c-11c2f2e8fbd2 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:12:10.026433 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46426016-f3d0-4b16-a83c-11c2f2e8fbd2 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:12:10.120212 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 655/1306] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 24 09:12:09 2022] POST /identity/v3/endpoints => generated 331 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:12:19.883844 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=30 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:12:20.115592 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:19:14.546590 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04be946-bb68-4fde-b5bc-33124921002b None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:19:14.552772 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04be946-bb68-4fde-b5bc-33124921002b None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:19:14.552772 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04be946-bb68-4fde-b5bc-33124921002b None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:19:14.554551 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c525224-114c-4b7c-b14d-88de81878230 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:19:14.555304 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c525224-114c-4b7c-b14d-88de81878230 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:19:14.555767 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c525224-114c-4b7c-b14d-88de81878230 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:19:14.561409 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 652/1307] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:19:14 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:19:14.665286 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-0c525224-114c-4b7c-b14d-88de81878230 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:19:14.667045 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-0c525224-114c-4b7c-b14d-88de81878230 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:19:14.803958 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 656/1308] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:19:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:19:15.116544 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18934f94-4719-4f8b-a5f8-940b2418a9c9 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:19:15.117201 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18934f94-4719-4f8b-a5f8-940b2418a9c9 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:19:15.117685 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18934f94-4719-4f8b-a5f8-940b2418a9c9 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:19:15.119498 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 653/1309] 10.222.0.16 () {54 vars in 965 bytes} [Wed Aug 24 09:19:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:19:15.133130 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421f7d54-caeb-4ed6-a0dc-f6e270fd2043 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:19:15.133766 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421f7d54-caeb-4ed6-a0dc-f6e270fd2043 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:19:15.134301 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421f7d54-caeb-4ed6-a0dc-f6e270fd2043 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:19:15.215968 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-421f7d54-caeb-4ed6-a0dc-f6e270fd2043 None None] MFA Rules not processed for user `8e5260e798404e8bb12b25ca8ea97052`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:19:15.218082 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-421f7d54-caeb-4ed6-a0dc-f6e270fd2043 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:19:15.240132 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:19:15.300086 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 657/1310] 10.222.0.16 () {58 vars in 1062 bytes} [Wed Aug 24 09:19:15 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:19:15.309185 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18934f94-4719-4f8b-a5f8-940b2418a9c9 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:19:15.311710 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-18934f94-4719-4f8b-a5f8-940b2418a9c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:19:15.434970 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18934f94-4719-4f8b-a5f8-940b2418a9c9 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:19:15.522272 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6812b9-191c-491f-aa4f-60efaabae375 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e5260e798404e8bb12b25ca8ea97052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:19:15.523330 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6812b9-191c-491f-aa4f-60efaabae375 service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:19:15.523881 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6812b9-191c-491f-aa4f-60efaabae375 service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:19:15.524369 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6812b9-191c-491f-aa4f-60efaabae375 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:19:15.526548 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-0d6812b9-191c-491f-aa4f-60efaabae375 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:19:15.595198 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:19:15.661653 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6812b9-191c-491f-aa4f-60efaabae375 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:19:15.663136 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6812b9-191c-491f-aa4f-60efaabae375 service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:19:15.687072 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 654/1311] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:19:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:19:17.910797 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e36cab7-479a-49d7-82d1-11f10a99344a None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:19:17.911360 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e36cab7-479a-49d7-82d1-11f10a99344a None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:19:17.911360 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e36cab7-479a-49d7-82d1-11f10a99344a None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:19:17.912974 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 658/1312] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:19:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:19:17.918475 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bb4af0-2b59-4167-8053-f671709afe4b None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:19:17.918913 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bb4af0-2b59-4167-8053-f671709afe4b None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:19:17.919160 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bb4af0-2b59-4167-8053-f671709afe4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:19:17.989877 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-f9bb4af0-2b59-4167-8053-f671709afe4b None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:19:17.991078 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-f9bb4af0-2b59-4167-8053-f671709afe4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:19:18.098293 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 655/1313] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:19:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:19:18.378412 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e36cab7-479a-49d7-82d1-11f10a99344a None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:19:18.379982 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-7e36cab7-479a-49d7-82d1-11f10a99344a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:19:18.482156 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e36cab7-479a-49d7-82d1-11f10a99344a None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:19:18.550571 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:19:18.553077 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8f8192-1b1b-4386-8a0b-dd4a3e150dda service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e5260e798404e8bb12b25ca8ea97052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:19:18.553676 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8f8192-1b1b-4386-8a0b-dd4a3e150dda service neutron] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:19:18.553953 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8f8192-1b1b-4386-8a0b-dd4a3e150dda service neutron] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:19:18.554222 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8f8192-1b1b-4386-8a0b-dd4a3e150dda service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:19:18.555835 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-8e8f8192-1b1b-4386-8a0b-dd4a3e150dda service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:19:18.660962 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8f8192-1b1b-4386-8a0b-dd4a3e150dda service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:19:18.662383 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8f8192-1b1b-4386-8a0b-dd4a3e150dda service neutron] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:19:18.680256 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 659/1314] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:19:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:19:21.026763 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fac173-686e-4b92-8f50-de2921d05291 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:19:21.027329 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fac173-686e-4b92-8f50-de2921d05291 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:19:21.027329 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fac173-686e-4b92-8f50-de2921d05291 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:19:21.028693 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 656/1315] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:19:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:19:21.034044 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d60775-84ec-4c9c-b39e-5359a6158d43 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:19:21.034461 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d60775-84ec-4c9c-b39e-5359a6158d43 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:19:21.034629 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d60775-84ec-4c9c-b39e-5359a6158d43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:19:21.099824 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-d0d60775-84ec-4c9c-b39e-5359a6158d43 None None] MFA Rules not processed for user `1925e7a247b64a06b5abece3c5687b12`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:19:21.101022 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-d0d60775-84ec-4c9c-b39e-5359a6158d43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:19:21.151367 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:19:21.195813 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 660/1316] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:19:21 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:19:21.484347 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fac173-686e-4b92-8f50-de2921d05291 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:19:21.485903 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-a0fac173-686e-4b92-8f50-de2921d05291 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:19:21.495690 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:19:21.588590 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fac173-686e-4b92-8f50-de2921d05291 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:19:21.659795 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f75651-4153-45bc-96fa-3f5ab8773353 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e5260e798404e8bb12b25ca8ea97052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:19:21.660400 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f75651-4153-45bc-96fa-3f5ab8773353 service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:19:21.660683 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f75651-4153-45bc-96fa-3f5ab8773353 service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:19:21.660958 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f75651-4153-45bc-96fa-3f5ab8773353 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:19:21.662556 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-20f75651-4153-45bc-96fa-3f5ab8773353 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:19:21.724260 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:19:21.771208 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f75651-4153-45bc-96fa-3f5ab8773353 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:19:21.772660 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f75651-4153-45bc-96fa-3f5ab8773353 service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:19:21.791087 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 657/1317] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:19:21 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:19:24.496266 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b3b98b-6d71-41c4-a30e-9cf609597f79 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:19:24.496850 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b3b98b-6d71-41c4-a30e-9cf609597f79 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:19:24.496850 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b3b98b-6d71-41c4-a30e-9cf609597f79 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:19:24.498652 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 661/1318] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:19:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:19:24.504031 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa9db20-d17e-4139-8c69-faa06495d5e4 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:19:24.504460 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa9db20-d17e-4139-8c69-faa06495d5e4 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:19:24.504594 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa9db20-d17e-4139-8c69-faa06495d5e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:19:24.568616 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-4fa9db20-d17e-4139-8c69-faa06495d5e4 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:19:24.569740 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-4fa9db20-d17e-4139-8c69-faa06495d5e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:19:24.662954 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 658/1319] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:19:24 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:19:24.945859 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b3b98b-6d71-41c4-a30e-9cf609597f79 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:19:24.947408 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-b2b3b98b-6d71-41c4-a30e-9cf609597f79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:19:25.049217 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b3b98b-6d71-41c4-a30e-9cf609597f79 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:19:25.121256 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5723bd8-be35-4e21-98ec-c95fdc829ad4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e5260e798404e8bb12b25ca8ea97052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:19:25.121861 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5723bd8-be35-4e21-98ec-c95fdc829ad4 service neutron] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:19:25.122135 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5723bd8-be35-4e21-98ec-c95fdc829ad4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:19:25.122401 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5723bd8-be35-4e21-98ec-c95fdc829ad4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:19:25.124010 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-a5723bd8-be35-4e21-98ec-c95fdc829ad4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:19:25.131636 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:19:25.229349 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5723bd8-be35-4e21-98ec-c95fdc829ad4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:19:25.230707 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5723bd8-be35-4e21-98ec-c95fdc829ad4 service neutron] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:19:25.248520 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 662/1320] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:19:24 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:19:28.210871 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5110d0-33cd-42f8-b9e5-60495776e3a3 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:19:28.211452 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5110d0-33cd-42f8-b9e5-60495776e3a3 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:19:28.211452 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5110d0-33cd-42f8-b9e5-60495776e3a3 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:19:28.213054 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 659/1321] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:19:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:19:28.218283 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cded5ee0-8b92-4b7b-afde-fcd1c9566047 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:19:28.218711 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cded5ee0-8b92-4b7b-afde-fcd1c9566047 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:19:28.218817 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cded5ee0-8b92-4b7b-afde-fcd1c9566047 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:19:28.286525 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-cded5ee0-8b92-4b7b-afde-fcd1c9566047 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:19:28.287778 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-cded5ee0-8b92-4b7b-afde-fcd1c9566047 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:19:28.350181 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:19:28.389756 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 663/1322] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:19:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:19:28.671949 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5110d0-33cd-42f8-b9e5-60495776e3a3 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:19:28.673616 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-6f5110d0-33cd-42f8-b9e5-60495776e3a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:19:28.693688 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:19:28.778866 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5110d0-33cd-42f8-b9e5-60495776e3a3 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:19:28.851531 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847be2ac-311a-4d27-b69c-acc603666da6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e5260e798404e8bb12b25ca8ea97052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:19:28.852175 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847be2ac-311a-4d27-b69c-acc603666da6 service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:19:28.852457 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847be2ac-311a-4d27-b69c-acc603666da6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:19:28.852733 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847be2ac-311a-4d27-b69c-acc603666da6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:19:28.854345 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-847be2ac-311a-4d27-b69c-acc603666da6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:19:28.925149 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:19:28.963451 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847be2ac-311a-4d27-b69c-acc603666da6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:19:28.964931 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847be2ac-311a-4d27-b69c-acc603666da6 service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:19:28.982907 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 660/1323] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:19:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:19:31.002663 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5578b2-02f4-4d38-b1bf-66407f6e09ea None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:19:31.003173 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5578b2-02f4-4d38-b1bf-66407f6e09ea None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:19:31.003173 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5578b2-02f4-4d38-b1bf-66407f6e09ea None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:19:31.004594 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 664/1324] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:19:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:19:31.010108 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0492a642-fdcd-4f6c-b88f-eba8118197bd None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:19:31.010569 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0492a642-fdcd-4f6c-b88f-eba8118197bd None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:19:31.010666 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0492a642-fdcd-4f6c-b88f-eba8118197bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:19:31.075863 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-0492a642-fdcd-4f6c-b88f-eba8118197bd None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:19:31.077088 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-0492a642-fdcd-4f6c-b88f-eba8118197bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:19:31.173091 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 661/1325] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:19:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:19:31.457057 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5578b2-02f4-4d38-b1bf-66407f6e09ea None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:19:31.458633 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-4d5578b2-02f4-4d38-b1bf-66407f6e09ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:19:31.571760 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5578b2-02f4-4d38-b1bf-66407f6e09ea None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:19:31.646710 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2855bd47-8b88-44fa-abb8-0a40b58bf7da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e5260e798404e8bb12b25ca8ea97052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:19:31.647320 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2855bd47-8b88-44fa-abb8-0a40b58bf7da service neutron] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:19:31.647621 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2855bd47-8b88-44fa-abb8-0a40b58bf7da service neutron] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:19:31.647899 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2855bd47-8b88-44fa-abb8-0a40b58bf7da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:19:31.649484 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-2855bd47-8b88-44fa-abb8-0a40b58bf7da service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:19:31.667309 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:19:31.759982 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2855bd47-8b88-44fa-abb8-0a40b58bf7da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:19:31.761431 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2855bd47-8b88-44fa-abb8-0a40b58bf7da service neutron] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:19:31.779869 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 665/1326] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:19:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:19:33.738500 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b284ca7a-9976-43cd-aa37-b6a8834e2a60 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:19:33.739019 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b284ca7a-9976-43cd-aa37-b6a8834e2a60 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:19:33.739019 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b284ca7a-9976-43cd-aa37-b6a8834e2a60 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:19:33.740427 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 662/1327] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:19:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:19:33.745847 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722ad2f8-9c92-4634-b2da-753baba47b6d None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:19:33.746274 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722ad2f8-9c92-4634-b2da-753baba47b6d None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:19:33.746390 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722ad2f8-9c92-4634-b2da-753baba47b6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:19:33.818022 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-722ad2f8-9c92-4634-b2da-753baba47b6d None None] MFA Rules not processed for user `1925e7a247b64a06b5abece3c5687b12`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:19:33.819228 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-722ad2f8-9c92-4634-b2da-753baba47b6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:19:33.882780 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:19:33.915318 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 666/1328] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:19:33 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:19:34.212218 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170333db-ae06-487e-99aa-40e446c6fd68 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:19:34.212499 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170333db-ae06-487e-99aa-40e446c6fd68 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:19:34.212769 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170333db-ae06-487e-99aa-40e446c6fd68 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:19:34.214114 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 663/1329] 10.222.0.16 () {54 vars in 965 bytes} [Wed Aug 24 09:19:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:19:34.220436 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2681ecb-e00e-4340-8f1a-14609e1cadf4 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:19:34.220718 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2681ecb-e00e-4340-8f1a-14609e1cadf4 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:19:34.221025 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2681ecb-e00e-4340-8f1a-14609e1cadf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:19:34.299238 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-f2681ecb-e00e-4340-8f1a-14609e1cadf4 None None] MFA Rules not processed for user `8e5260e798404e8bb12b25ca8ea97052`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:19:34.300491 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-f2681ecb-e00e-4340-8f1a-14609e1cadf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:19:34.379499 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 667/1330] 10.222.0.16 () {58 vars in 1062 bytes} [Wed Aug 24 09:19:34 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:19:34.388453 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170333db-ae06-487e-99aa-40e446c6fd68 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:19:34.390131 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-170333db-ae06-487e-99aa-40e446c6fd68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:19:34.424609 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:19:34.506385 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170333db-ae06-487e-99aa-40e446c6fd68 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:19:34.583214 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68eeaac7-1f89-4038-b537-55e858b49854 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e5260e798404e8bb12b25ca8ea97052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:19:34.583907 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68eeaac7-1f89-4038-b537-55e858b49854 service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:19:34.584190 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68eeaac7-1f89-4038-b537-55e858b49854 service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:19:34.584463 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68eeaac7-1f89-4038-b537-55e858b49854 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:19:34.586087 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-68eeaac7-1f89-4038-b537-55e858b49854 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:19:34.664601 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:19:34.697463 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68eeaac7-1f89-4038-b537-55e858b49854 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:19:34.699059 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68eeaac7-1f89-4038-b537-55e858b49854 service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:19:34.718082 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 664/1331] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:19:34 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:19:37.461641 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877f01f0-5717-487d-ac0b-ed48e4c41b15 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:19:37.462449 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877f01f0-5717-487d-ac0b-ed48e4c41b15 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:19:37.462449 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877f01f0-5717-487d-ac0b-ed48e4c41b15 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:19:37.463807 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 668/1332] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:19:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 09:19:37.469073 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0dfc6d-b8c3-4cff-91b1-ef5298eb09f4 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:19:37.469473 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0dfc6d-b8c3-4cff-91b1-ef5298eb09f4 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:19:37.469631 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0dfc6d-b8c3-4cff-91b1-ef5298eb09f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:19:37.536000 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-8e0dfc6d-b8c3-4cff-91b1-ef5298eb09f4 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:19:37.537204 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-8e0dfc6d-b8c3-4cff-91b1-ef5298eb09f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:19:37.631527 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 665/1333] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:19:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:19:37.912064 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877f01f0-5717-487d-ac0b-ed48e4c41b15 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:19:37.913640 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-877f01f0-5717-487d-ac0b-ed48e4c41b15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:19:37.983540 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:19:38.016205 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877f01f0-5717-487d-ac0b-ed48e4c41b15 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:19:38.090992 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391b5bf5-4230-485d-a97f-1032704bb4fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e5260e798404e8bb12b25ca8ea97052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:19:38.092185 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391b5bf5-4230-485d-a97f-1032704bb4fd service neutron] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:19:38.092472 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391b5bf5-4230-485d-a97f-1032704bb4fd service neutron] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:19:38.092746 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391b5bf5-4230-485d-a97f-1032704bb4fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:19:38.094564 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-391b5bf5-4230-485d-a97f-1032704bb4fd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:19:38.200979 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391b5bf5-4230-485d-a97f-1032704bb4fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:19:38.202332 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391b5bf5-4230-485d-a97f-1032704bb4fd service neutron] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:19:38.212457 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:19:38.220977 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 669/1334] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:19:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:19:40.161774 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ea0202-7244-437d-b14f-a405e19f0b55 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:19:40.162401 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ea0202-7244-437d-b14f-a405e19f0b55 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:19:40.162401 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ea0202-7244-437d-b14f-a405e19f0b55 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:19:40.163684 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 666/1335] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:19:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:19:40.168769 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4ea0e9-ac49-430f-b397-13e312da9546 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:19:40.169217 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4ea0e9-ac49-430f-b397-13e312da9546 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:19:40.169326 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4ea0e9-ac49-430f-b397-13e312da9546 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:19:40.234116 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-8f4ea0e9-ac49-430f-b397-13e312da9546 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:19:40.235311 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-8f4ea0e9-ac49-430f-b397-13e312da9546 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:19:40.334493 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 670/1336] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:19:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:19:40.619041 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ea0202-7244-437d-b14f-a405e19f0b55 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:19:40.620701 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-d1ea0202-7244-437d-b14f-a405e19f0b55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:19:40.660596 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:19:40.723319 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ea0202-7244-437d-b14f-a405e19f0b55 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:19:40.793875 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5059c94-eb68-4fb2-ae2e-f630345f4f4e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e5260e798404e8bb12b25ca8ea97052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:19:40.794486 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5059c94-eb68-4fb2-ae2e-f630345f4f4e service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:19:40.795292 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5059c94-eb68-4fb2-ae2e-f630345f4f4e service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:19:40.795563 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5059c94-eb68-4fb2-ae2e-f630345f4f4e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:19:40.797168 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-d5059c94-eb68-4fb2-ae2e-f630345f4f4e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:19:40.880052 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:19:40.904998 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5059c94-eb68-4fb2-ae2e-f630345f4f4e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:19:40.906411 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5059c94-eb68-4fb2-ae2e-f630345f4f4e service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:19:40.924169 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 667/1337] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:19:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:19:43.495843 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea0210b-42f6-4d0f-bdbc-fe91c2303a5a None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:19:43.496398 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea0210b-42f6-4d0f-bdbc-fe91c2303a5a None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:19:43.496398 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea0210b-42f6-4d0f-bdbc-fe91c2303a5a None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:19:43.497759 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 671/1338] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:19:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:19:43.503066 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d931fddd-f198-4031-978f-e75982b19c9b None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:19:43.503471 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d931fddd-f198-4031-978f-e75982b19c9b None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:19:43.503662 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d931fddd-f198-4031-978f-e75982b19c9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:19:43.568884 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-d931fddd-f198-4031-978f-e75982b19c9b None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:19:43.570019 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-d931fddd-f198-4031-978f-e75982b19c9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:19:43.665855 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 668/1339] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:19:43 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:19:43.946106 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea0210b-42f6-4d0f-bdbc-fe91c2303a5a None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:19:43.947713 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-dea0210b-42f6-4d0f-bdbc-fe91c2303a5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:19:44.024381 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:19:44.051421 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea0210b-42f6-4d0f-bdbc-fe91c2303a5a None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:19:44.125321 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d7564d-31b7-42fa-8081-61b7f828a101 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e5260e798404e8bb12b25ca8ea97052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:19:44.125933 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d7564d-31b7-42fa-8081-61b7f828a101 service neutron] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:19:44.126218 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d7564d-31b7-42fa-8081-61b7f828a101 service neutron] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:19:44.126494 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d7564d-31b7-42fa-8081-61b7f828a101 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:19:44.128132 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-f6d7564d-31b7-42fa-8081-61b7f828a101 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:19:44.235481 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d7564d-31b7-42fa-8081-61b7f828a101 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:19:44.236876 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d7564d-31b7-42fa-8081-61b7f828a101 service neutron] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:19:44.255094 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 672/1340] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:19:43 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:19:48.470097 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75df4f0e-e229-4af9-9b00-ca309a22853c None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:19:48.470632 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75df4f0e-e229-4af9-9b00-ca309a22853c None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:19:48.470632 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75df4f0e-e229-4af9-9b00-ca309a22853c None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:19:48.472023 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 669/1341] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:19:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:19:48.477546 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a6f08b-86e1-46e1-8fba-817855a67a32 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:19:48.478009 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a6f08b-86e1-46e1-8fba-817855a67a32 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:19:48.478106 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a6f08b-86e1-46e1-8fba-817855a67a32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:19:48.483756 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:19:48.545672 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-f7a6f08b-86e1-46e1-8fba-817855a67a32 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:19:48.546890 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-f7a6f08b-86e1-46e1-8fba-817855a67a32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:19:48.645151 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 673/1342] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:19:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:19:48.925627 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75df4f0e-e229-4af9-9b00-ca309a22853c None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:19:48.927205 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-75df4f0e-e229-4af9-9b00-ca309a22853c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:19:48.975171 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:19:49.030929 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75df4f0e-e229-4af9-9b00-ca309a22853c None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:19:49.104120 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992f8e2f-5792-4ff0-ab8f-96661d5d9fb9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e5260e798404e8bb12b25ca8ea97052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:19:49.104731 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992f8e2f-5792-4ff0-ab8f-96661d5d9fb9 service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:19:49.105005 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992f8e2f-5792-4ff0-ab8f-96661d5d9fb9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:19:49.105275 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992f8e2f-5792-4ff0-ab8f-96661d5d9fb9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:19:49.106859 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-992f8e2f-5792-4ff0-ab8f-96661d5d9fb9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:19:49.197906 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:19:49.214376 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992f8e2f-5792-4ff0-ab8f-96661d5d9fb9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:19:49.215837 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992f8e2f-5792-4ff0-ab8f-96661d5d9fb9 service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:19:49.233612 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 670/1343] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:19:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:19:52.182840 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e48f4ca-a5dc-4e31-9330-3e69aedc0a44 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:19:52.183353 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e48f4ca-a5dc-4e31-9330-3e69aedc0a44 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:19:52.183353 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e48f4ca-a5dc-4e31-9330-3e69aedc0a44 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:19:52.184674 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 674/1344] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:19:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:19:52.190161 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e7a806-997d-44f8-aba0-89d14b2ab0c8 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:19:52.190580 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e7a806-997d-44f8-aba0-89d14b2ab0c8 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:19:52.190776 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e7a806-997d-44f8-aba0-89d14b2ab0c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:19:52.257493 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-c1e7a806-997d-44f8-aba0-89d14b2ab0c8 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:19:52.258686 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-c1e7a806-997d-44f8-aba0-89d14b2ab0c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:19:52.358262 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 671/1345] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:19:52 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:19:52.648039 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e48f4ca-a5dc-4e31-9330-3e69aedc0a44 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:19:52.649643 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-2e48f4ca-a5dc-4e31-9330-3e69aedc0a44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:19:52.738159 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:19:52.754800 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e48f4ca-a5dc-4e31-9330-3e69aedc0a44 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:19:52.834506 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7686ee7a-9b54-4f8e-a915-0af12efeb3c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e5260e798404e8bb12b25ca8ea97052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:19:52.835207 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7686ee7a-9b54-4f8e-a915-0af12efeb3c8 service neutron] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:19:52.835414 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7686ee7a-9b54-4f8e-a915-0af12efeb3c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:19:52.835727 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7686ee7a-9b54-4f8e-a915-0af12efeb3c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:19:52.837586 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-7686ee7a-9b54-4f8e-a915-0af12efeb3c8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:19:52.949549 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7686ee7a-9b54-4f8e-a915-0af12efeb3c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:19:52.951018 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7686ee7a-9b54-4f8e-a915-0af12efeb3c8 service neutron] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:19:52.970353 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 675/1346] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:19:52 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:19:56.955974 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e9e681-c440-42e7-bbfd-f7697b9262af None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:19:56.956552 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e9e681-c440-42e7-bbfd-f7697b9262af None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:19:56.956552 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e9e681-c440-42e7-bbfd-f7697b9262af None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:19:56.957908 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 672/1347] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:19:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:19:56.963942 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1683be98-5bb7-43d1-833f-78ea7dac5089 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:19:56.964460 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1683be98-5bb7-43d1-833f-78ea7dac5089 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:19:56.964568 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1683be98-5bb7-43d1-833f-78ea7dac5089 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:19:56.980628 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:19:57.035153 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-1683be98-5bb7-43d1-833f-78ea7dac5089 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:19:57.036432 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-1683be98-5bb7-43d1-833f-78ea7dac5089 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:19:57.144712 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 676/1348] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:19:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:19:57.436011 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e9e681-c440-42e7-bbfd-f7697b9262af None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:19:57.437657 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-92e9e681-c440-42e7-bbfd-f7697b9262af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:19:57.496815 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:19:57.541306 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e9e681-c440-42e7-bbfd-f7697b9262af None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:19:57.612828 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed400978-b735-480f-b11f-52d1df2ce278 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e5260e798404e8bb12b25ca8ea97052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:19:57.613359 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed400978-b735-480f-b11f-52d1df2ce278 service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:19:57.613632 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed400978-b735-480f-b11f-52d1df2ce278 service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:19:57.613898 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed400978-b735-480f-b11f-52d1df2ce278 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:19:57.615484 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-ed400978-b735-480f-b11f-52d1df2ce278 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:19:57.718947 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:19:57.722432 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed400978-b735-480f-b11f-52d1df2ce278 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:19:57.730988 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed400978-b735-480f-b11f-52d1df2ce278 service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:19:57.749836 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 673/1349] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:19:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:20:02.764310 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ebea97-668c-44c9-bfb8-7c6fb913259f None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:20:02.765430 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ebea97-668c-44c9-bfb8-7c6fb913259f None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:20:02.765430 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ebea97-668c-44c9-bfb8-7c6fb913259f None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:20:02.766415 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 677/1350] 10.222.0.16 () {54 vars in 921 bytes} [Wed Aug 24 09:20:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:20:02.786399 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfef6934-edea-4ae4-a4ec-5c2177869fa5 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:20:02.786844 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfef6934-edea-4ae4-a4ec-5c2177869fa5 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:20:02.787043 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfef6934-edea-4ae4-a4ec-5c2177869fa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:20:02.912117 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-bfef6934-edea-4ae4-a4ec-5c2177869fa5 None None] MFA Rules not processed for user `80cada22039d4f11935412ee578acd05`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:20:02.913458 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-bfef6934-edea-4ae4-a4ec-5c2177869fa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:20:03.032220 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 674/1351] 10.222.0.16 () {58 vars in 1018 bytes} [Wed Aug 24 09:20:02 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:20:07.146863 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=59 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:20:07.485090 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb55679-eaff-4f81-9729-0c5b181bafa0 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:20:07.485267 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb55679-eaff-4f81-9729-0c5b181bafa0 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:20:07.485722 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb55679-eaff-4f81-9729-0c5b181bafa0 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:20:07.487477 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 678/1352] 10.222.0.16 () {54 vars in 921 bytes} [Wed Aug 24 09:20:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:20:07.492781 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4e0360-989d-46a3-9391-483a21ec0c0d None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:20:07.493679 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4e0360-989d-46a3-9391-483a21ec0c0d None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:20:07.494175 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4e0360-989d-46a3-9391-483a21ec0c0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:20:07.550278 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:20:07.573268 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-9d4e0360-989d-46a3-9391-483a21ec0c0d None None] MFA Rules not processed for user `80cada22039d4f11935412ee578acd05`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:20:07.574755 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-9d4e0360-989d-46a3-9391-483a21ec0c0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:20:07.690163 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 675/1353] 10.222.0.16 () {58 vars in 1018 bytes} [Wed Aug 24 09:20:07 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:20:09.105048 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4a066c-f3dd-4c2a-9f67-bdd2b9c80125 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:20:09.105597 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4a066c-f3dd-4c2a-9f67-bdd2b9c80125 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:20:09.105597 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4a066c-f3dd-4c2a-9f67-bdd2b9c80125 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:20:09.107084 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 679/1354] 10.222.0.16 () {54 vars in 921 bytes} [Wed Aug 24 09:20:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:20:09.112313 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeac21c-f092-4cfa-8007-e0e26ad39451 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:20:09.112724 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeac21c-f092-4cfa-8007-e0e26ad39451 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:20:09.112993 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeac21c-f092-4cfa-8007-e0e26ad39451 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:20:09.189361 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-cbeac21c-f092-4cfa-8007-e0e26ad39451 None None] MFA Rules not processed for user `80cada22039d4f11935412ee578acd05`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:20:09.190554 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-cbeac21c-f092-4cfa-8007-e0e26ad39451 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:20:09.231833 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:20:09.298994 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 676/1355] 10.222.0.16 () {58 vars in 1018 bytes} [Wed Aug 24 09:20:09 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 09:20:12.636729 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf66a61-af0d-4c0a-88a4-598a4c84b493 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:20:12.638228 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf66a61-af0d-4c0a-88a4-598a4c84b493 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:20:12.638665 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf66a61-af0d-4c0a-88a4-598a4c84b493 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:20:12.640218 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 680/1356] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:20:12 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:20:12.646020 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1365848e-ee24-4051-89ba-1670c7da3b50 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:20:12.646472 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1365848e-ee24-4051-89ba-1670c7da3b50 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:20:12.646703 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1365848e-ee24-4051-89ba-1670c7da3b50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:20:12.735283 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-1365848e-ee24-4051-89ba-1670c7da3b50 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:20:12.736657 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-1365848e-ee24-4051-89ba-1670c7da3b50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:20:12.869651 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 677/1357] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:20:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:20:13.166953 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6e6cb2-908f-4183-b159-aa0dc603f8d2 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:20:13.167495 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6e6cb2-908f-4183-b159-aa0dc603f8d2 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:20:13.168008 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6e6cb2-908f-4183-b159-aa0dc603f8d2 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:20:13.169771 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 681/1358] 10.222.0.16 () {54 vars in 962 bytes} [Wed Aug 24 09:20:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:20:13.183977 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6d075b-f919-48ee-a40b-2fafacf9b972 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:20:13.183977 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6d075b-f919-48ee-a40b-2fafacf9b972 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:20:13.183977 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6d075b-f919-48ee-a40b-2fafacf9b972 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:20:13.195541 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:20:13.258786 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-0e6d075b-f919-48ee-a40b-2fafacf9b972 None None] MFA Rules not processed for user `1f055f7dc51d43bda9b315ca67f11d58`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:20:13.260942 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-0e6d075b-f919-48ee-a40b-2fafacf9b972 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:20:13.405490 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 678/1359] 10.222.0.16 () {58 vars in 1059 bytes} [Wed Aug 24 09:20:13 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:20:13.413806 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6e6cb2-908f-4183-b159-aa0dc603f8d2 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:20:13.415784 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-2d6e6cb2-908f-4183-b159-aa0dc603f8d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:20:13.576058 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6e6cb2-908f-4183-b159-aa0dc603f8d2 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:20:13.658360 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:20:13.706211 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a438388-c8de-4fd9-adad-473709c0f10a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f055f7dc51d43bda9b315ca67f11d58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:20:13.707252 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a438388-c8de-4fd9-adad-473709c0f10a service nova] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:20:13.707707 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a438388-c8de-4fd9-adad-473709c0f10a service nova] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:20:13.708215 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a438388-c8de-4fd9-adad-473709c0f10a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:20:13.710149 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-7a438388-c8de-4fd9-adad-473709c0f10a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:20:13.812594 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a438388-c8de-4fd9-adad-473709c0f10a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:20:13.814239 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a438388-c8de-4fd9-adad-473709c0f10a service nova] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:20:13.833778 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 682/1360] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:20:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:20:15.959986 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447bdeeb-f6a4-4453-95dc-a6cc96745fd3 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:20:15.960533 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447bdeeb-f6a4-4453-95dc-a6cc96745fd3 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:20:15.960533 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447bdeeb-f6a4-4453-95dc-a6cc96745fd3 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:20:15.961963 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 679/1361] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:20:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:20:15.967916 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af693a97-8eb5-4afd-ac8a-2e97cce857d6 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:20:15.968317 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af693a97-8eb5-4afd-ac8a-2e97cce857d6 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:20:15.968488 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af693a97-8eb5-4afd-ac8a-2e97cce857d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:20:16.036410 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-af693a97-8eb5-4afd-ac8a-2e97cce857d6 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:20:16.037625 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-af693a97-8eb5-4afd-ac8a-2e97cce857d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:20:16.075265 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:20:16.142146 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 683/1362] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:20:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:20:16.429650 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447bdeeb-f6a4-4453-95dc-a6cc96745fd3 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:20:16.435218 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-447bdeeb-f6a4-4453-95dc-a6cc96745fd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:20:16.504055 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:20:16.604246 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447bdeeb-f6a4-4453-95dc-a6cc96745fd3 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:20:16.727295 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0342ea07-179a-4b0c-be20-09965031a89d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f055f7dc51d43bda9b315ca67f11d58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:20:16.728070 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0342ea07-179a-4b0c-be20-09965031a89d service nova] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:20:16.728416 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0342ea07-179a-4b0c-be20-09965031a89d service nova] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:20:16.728765 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0342ea07-179a-4b0c-be20-09965031a89d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:20:16.730486 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-0342ea07-179a-4b0c-be20-09965031a89d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:20:16.757463 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:20:16.837510 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0342ea07-179a-4b0c-be20-09965031a89d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:20:16.839974 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0342ea07-179a-4b0c-be20-09965031a89d service nova] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:20:16.860078 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 680/1363] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:20:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:20:19.244693 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbaac65-2814-4f2c-ada9-0318a810b44c None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:20:19.245278 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbaac65-2814-4f2c-ada9-0318a810b44c None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:20:19.245278 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbaac65-2814-4f2c-ada9-0318a810b44c None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:20:19.247139 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 684/1364] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:20:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:20:19.252777 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35fce5c-351e-40ec-9cff-bc5a3aa8e57a None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:20:19.253225 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35fce5c-351e-40ec-9cff-bc5a3aa8e57a None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:20:19.253557 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35fce5c-351e-40ec-9cff-bc5a3aa8e57a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:20:19.327522 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-d35fce5c-351e-40ec-9cff-bc5a3aa8e57a None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:20:19.331488 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-d35fce5c-351e-40ec-9cff-bc5a3aa8e57a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:20:19.432866 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:20:19.439623 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 681/1365] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:20:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:20:19.765884 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11378287-5df1-4095-9ad7-6427f66da9fc None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:20:19.766216 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11378287-5df1-4095-9ad7-6427f66da9fc None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:20:19.766493 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11378287-5df1-4095-9ad7-6427f66da9fc None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:20:19.767954 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 685/1366] 10.222.0.16 () {54 vars in 962 bytes} [Wed Aug 24 09:20:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:20:19.775033 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d363e1c6-2e44-4b26-865d-daa0c1818624 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:20:19.775431 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d363e1c6-2e44-4b26-865d-daa0c1818624 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:20:19.775833 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d363e1c6-2e44-4b26-865d-daa0c1818624 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:20:19.858423 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-d363e1c6-2e44-4b26-865d-daa0c1818624 None None] MFA Rules not processed for user `1f055f7dc51d43bda9b315ca67f11d58`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:20:19.859819 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-d363e1c6-2e44-4b26-865d-daa0c1818624 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:20:19.999166 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 682/1367] 10.222.0.16 () {58 vars in 1059 bytes} [Wed Aug 24 09:20:19 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:20:20.006984 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11378287-5df1-4095-9ad7-6427f66da9fc None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:20:20.008569 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-11378287-5df1-4095-9ad7-6427f66da9fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:20:20.177181 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11378287-5df1-4095-9ad7-6427f66da9fc None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:20:20.199275 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:20:20.302664 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973837b1-c187-491f-83c8-a414ac78e3e0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f055f7dc51d43bda9b315ca67f11d58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:20:20.304139 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973837b1-c187-491f-83c8-a414ac78e3e0 service nova] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:20:20.304632 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973837b1-c187-491f-83c8-a414ac78e3e0 service nova] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:20:20.305109 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973837b1-c187-491f-83c8-a414ac78e3e0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:20:20.306947 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-973837b1-c187-491f-83c8-a414ac78e3e0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:20:20.407122 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973837b1-c187-491f-83c8-a414ac78e3e0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:20:20.409020 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973837b1-c187-491f-83c8-a414ac78e3e0 service nova] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:20:20.428638 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 686/1368] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:20:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:20:20.806503 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90bed8a-c554-4913-964e-0d1b88114fa9 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:20:20.807077 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90bed8a-c554-4913-964e-0d1b88114fa9 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:20:20.807077 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90bed8a-c554-4913-964e-0d1b88114fa9 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:20:20.808429 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 683/1369] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:20:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:20:20.814164 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bd8cf1-d71d-4bf6-a7d1-74d0845d7163 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:20:20.814447 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bd8cf1-d71d-4bf6-a7d1-74d0845d7163 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:20:20.814722 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bd8cf1-d71d-4bf6-a7d1-74d0845d7163 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:20:20.858031 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:20:20.892416 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-b2bd8cf1-d71d-4bf6-a7d1-74d0845d7163 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:20:20.893826 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-b2bd8cf1-d71d-4bf6-a7d1-74d0845d7163 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:20:20.998103 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 687/1370] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:20:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:20:21.583295 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e8abbf-d11c-46a3-8f9e-b2dd622197c4 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:20:21.583671 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e8abbf-d11c-46a3-8f9e-b2dd622197c4 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:20:21.584013 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e8abbf-d11c-46a3-8f9e-b2dd622197c4 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:20:21.585565 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 684/1371] 10.222.0.16 () {54 vars in 964 bytes} [Wed Aug 24 09:20:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:20:21.591561 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b993ae-03d4-4a31-96b8-7a2bd591b61a None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:20:21.592040 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b993ae-03d4-4a31-96b8-7a2bd591b61a None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:20:21.592325 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b993ae-03d4-4a31-96b8-7a2bd591b61a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:20:21.672706 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-17b993ae-03d4-4a31-96b8-7a2bd591b61a None None] MFA Rules not processed for user `05c32b83577c46488906fe409a790b24`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:20:21.674733 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-17b993ae-03d4-4a31-96b8-7a2bd591b61a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:20:21.727158 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:20:21.759648 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 688/1372] 10.222.0.16 () {58 vars in 1061 bytes} [Wed Aug 24 09:20:21 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:20:21.767970 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e8abbf-d11c-46a3-8f9e-b2dd622197c4 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:20:21.769728 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-21e8abbf-d11c-46a3-8f9e-b2dd622197c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:20:21.823055 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:20:21.877257 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e8abbf-d11c-46a3-8f9e-b2dd622197c4 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:20:21.954136 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cbbc22-3ba6-4993-b33d-503aa9c4b080 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c32b83577c46488906fe409a790b24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:20:21.954765 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cbbc22-3ba6-4993-b33d-503aa9c4b080 service cinder] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:20:21.955049 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cbbc22-3ba6-4993-b33d-503aa9c4b080 service cinder] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:20:21.955324 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cbbc22-3ba6-4993-b33d-503aa9c4b080 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:20:21.956982 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-45cbbc22-3ba6-4993-b33d-503aa9c4b080 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:20:22.080767 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:20:22.095693 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cbbc22-3ba6-4993-b33d-503aa9c4b080 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:20:22.097130 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cbbc22-3ba6-4993-b33d-503aa9c4b080 service cinder] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:20:22.122455 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 685/1373] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:20:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:20:22.588704 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e6e2d2-8428-443d-b272-414e394d7558 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:20:22.588895 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e6e2d2-8428-443d-b272-414e394d7558 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:20:22.589169 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e6e2d2-8428-443d-b272-414e394d7558 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:20:22.590552 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 689/1374] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:20:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:20:22.595734 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3b985e-1fac-447d-9c9f-52466357d94e None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:20:22.596089 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3b985e-1fac-447d-9c9f-52466357d94e None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:20:22.596398 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3b985e-1fac-447d-9c9f-52466357d94e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:20:22.664132 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-5b3b985e-1fac-447d-9c9f-52466357d94e None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:20:22.665340 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-5b3b985e-1fac-447d-9c9f-52466357d94e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:20:22.774214 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 686/1375] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:20:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:20:23.073306 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e6e2d2-8428-443d-b272-414e394d7558 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:20:23.075337 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-c4e6e2d2-8428-443d-b272-414e394d7558 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:20:23.219563 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e6e2d2-8428-443d-b272-414e394d7558 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:20:23.270605 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:20:23.333258 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ade66fa-1d24-4783-9af4-b2a7c36ba08c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f055f7dc51d43bda9b315ca67f11d58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:20:23.333980 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ade66fa-1d24-4783-9af4-b2a7c36ba08c service nova] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:20:23.334448 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ade66fa-1d24-4783-9af4-b2a7c36ba08c service nova] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:20:23.334880 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ade66fa-1d24-4783-9af4-b2a7c36ba08c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:20:23.337019 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-3ade66fa-1d24-4783-9af4-b2a7c36ba08c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:20:23.428296 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ade66fa-1d24-4783-9af4-b2a7c36ba08c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:20:23.429881 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ade66fa-1d24-4783-9af4-b2a7c36ba08c service nova] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:20:23.448815 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 690/1376] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:20:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:20:25.733824 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa32134-86c2-4a04-abfa-f040e988409a None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:20:25.734399 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa32134-86c2-4a04-abfa-f040e988409a None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:20:25.739960 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa32134-86c2-4a04-abfa-f040e988409a None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:20:25.741435 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 687/1377] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:20:25 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:20:25.747082 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d03946-8e17-4def-bb57-ca20016a8d31 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:20:25.747919 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d03946-8e17-4def-bb57-ca20016a8d31 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:20:25.748198 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d03946-8e17-4def-bb57-ca20016a8d31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:20:25.823200 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-58d03946-8e17-4def-bb57-ca20016a8d31 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:20:25.828633 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-58d03946-8e17-4def-bb57-ca20016a8d31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:20:25.842847 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:20:25.969640 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 691/1378] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:20:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:20:26.260089 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa32134-86c2-4a04-abfa-f040e988409a None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:20:26.261741 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-ffa32134-86c2-4a04-abfa-f040e988409a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:20:26.336371 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:20:26.429312 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa32134-86c2-4a04-abfa-f040e988409a None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:20:26.559349 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88b2d55-4e39-4b0d-9b52-93f77886f689 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f055f7dc51d43bda9b315ca67f11d58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:20:26.560171 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88b2d55-4e39-4b0d-9b52-93f77886f689 service nova] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:20:26.560566 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88b2d55-4e39-4b0d-9b52-93f77886f689 service nova] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:20:26.560942 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88b2d55-4e39-4b0d-9b52-93f77886f689 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:20:26.562656 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-c88b2d55-4e39-4b0d-9b52-93f77886f689 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:20:26.582409 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:20:26.662719 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88b2d55-4e39-4b0d-9b52-93f77886f689 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:20:26.664506 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88b2d55-4e39-4b0d-9b52-93f77886f689 service nova] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:20:26.684243 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 688/1379] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:20:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:20:28.732770 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74260e55-603e-4f17-8c19-132dd92d041b None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:20:28.733371 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74260e55-603e-4f17-8c19-132dd92d041b None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:20:28.733371 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74260e55-603e-4f17-8c19-132dd92d041b None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:20:28.735055 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 692/1380] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:20:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:20:28.741913 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da61554-23b1-403c-b3a6-43fb5649fbf1 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:20:28.742501 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da61554-23b1-403c-b3a6-43fb5649fbf1 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:20:28.742588 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da61554-23b1-403c-b3a6-43fb5649fbf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:20:28.820516 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-1da61554-23b1-403c-b3a6-43fb5649fbf1 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:20:28.821788 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-1da61554-23b1-403c-b3a6-43fb5649fbf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:20:28.922516 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:20:28.929416 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 689/1381] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:20:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:20:29.227765 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74260e55-603e-4f17-8c19-132dd92d041b None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:20:29.229298 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-74260e55-603e-4f17-8c19-132dd92d041b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:20:29.389114 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:20:29.399952 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74260e55-603e-4f17-8c19-132dd92d041b None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:20:29.521681 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc5764a-3967-4908-89b8-ff0b93ec8991 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f055f7dc51d43bda9b315ca67f11d58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:20:29.522293 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc5764a-3967-4908-89b8-ff0b93ec8991 service nova] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:20:29.522552 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc5764a-3967-4908-89b8-ff0b93ec8991 service nova] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:20:29.522944 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc5764a-3967-4908-89b8-ff0b93ec8991 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:20:29.524518 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-ecc5764a-3967-4908-89b8-ff0b93ec8991 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:20:29.621752 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc5764a-3967-4908-89b8-ff0b93ec8991 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:20:29.623378 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc5764a-3967-4908-89b8-ff0b93ec8991 service nova] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:20:29.633815 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:20:29.642205 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 693/1382] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:20:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:20:31.891914 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2150103f-f3b3-47f8-aba6-a066fd58cb36 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:20:31.892499 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2150103f-f3b3-47f8-aba6-a066fd58cb36 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:20:31.892499 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2150103f-f3b3-47f8-aba6-a066fd58cb36 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:20:31.893812 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 690/1383] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:20:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:20:31.899257 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc837cd7-785a-4d54-998b-717167188a03 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:20:31.899720 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc837cd7-785a-4d54-998b-717167188a03 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:20:31.899837 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc837cd7-785a-4d54-998b-717167188a03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:20:31.980195 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-dc837cd7-785a-4d54-998b-717167188a03 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:20:31.981978 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-dc837cd7-785a-4d54-998b-717167188a03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:20:32.092233 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 694/1384] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:20:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:20:32.427498 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2150103f-f3b3-47f8-aba6-a066fd58cb36 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:20:32.430362 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-2150103f-f3b3-47f8-aba6-a066fd58cb36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:20:32.583901 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2150103f-f3b3-47f8-aba6-a066fd58cb36 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:20:32.657193 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:20:32.705946 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ddb85d-b1eb-4d9b-82c5-082cb04413e7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f055f7dc51d43bda9b315ca67f11d58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:20:32.706563 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ddb85d-b1eb-4d9b-82c5-082cb04413e7 service nova] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:20:32.706841 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ddb85d-b1eb-4d9b-82c5-082cb04413e7 service nova] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:20:32.707120 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ddb85d-b1eb-4d9b-82c5-082cb04413e7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:20:32.708783 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-f3ddb85d-b1eb-4d9b-82c5-082cb04413e7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:20:32.802016 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ddb85d-b1eb-4d9b-82c5-082cb04413e7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:20:32.803645 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ddb85d-b1eb-4d9b-82c5-082cb04413e7 service nova] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:20:32.822829 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 691/1385] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:20:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:20:35.057427 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e58dbc1-552a-4f80-bf90-2f2efec28d85 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:20:35.058009 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e58dbc1-552a-4f80-bf90-2f2efec28d85 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:20:35.058009 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e58dbc1-552a-4f80-bf90-2f2efec28d85 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:20:35.059991 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 695/1386] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:20:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:20:35.066807 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60a1142-29db-4a1a-a42f-86dc650e346b None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:20:35.067342 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60a1142-29db-4a1a-a42f-86dc650e346b None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:20:35.067843 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60a1142-29db-4a1a-a42f-86dc650e346b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:20:35.174138 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-e60a1142-29db-4a1a-a42f-86dc650e346b None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:20:35.176358 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-e60a1142-29db-4a1a-a42f-86dc650e346b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:20:35.220457 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:20:35.296786 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 692/1387] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:20:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:20:35.805380 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e58dbc1-552a-4f80-bf90-2f2efec28d85 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:20:35.806909 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-2e58dbc1-552a-4f80-bf90-2f2efec28d85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:20:35.890046 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:20:35.965685 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e58dbc1-552a-4f80-bf90-2f2efec28d85 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:20:36.081118 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0d0259-47b6-4096-8f8a-3c53b5f9862a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f055f7dc51d43bda9b315ca67f11d58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:20:36.081770 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0d0259-47b6-4096-8f8a-3c53b5f9862a service nova] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:20:36.081894 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0d0259-47b6-4096-8f8a-3c53b5f9862a service nova] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:20:36.082151 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0d0259-47b6-4096-8f8a-3c53b5f9862a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:20:36.083819 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-9b0d0259-47b6-4096-8f8a-3c53b5f9862a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:20:36.124482 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:20:36.177688 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0d0259-47b6-4096-8f8a-3c53b5f9862a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:20:36.179326 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0d0259-47b6-4096-8f8a-3c53b5f9862a service nova] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:20:36.197875 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 696/1388] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:20:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:20:37.720207 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9698b503-22c8-416b-8232-2f265f971f7c None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:20:37.720759 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9698b503-22c8-416b-8232-2f265f971f7c None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:20:37.720759 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9698b503-22c8-416b-8232-2f265f971f7c None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:20:37.722020 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 693/1389] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:20:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:20:37.727256 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc962c21-e18e-4770-859c-49f3eb2cda4e None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:20:37.727688 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc962c21-e18e-4770-859c-49f3eb2cda4e None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:20:37.727837 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc962c21-e18e-4770-859c-49f3eb2cda4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:20:37.798213 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-bc962c21-e18e-4770-859c-49f3eb2cda4e None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:20:37.799433 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-bc962c21-e18e-4770-859c-49f3eb2cda4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:20:37.899003 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 697/1390] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:20:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:20:38.229122 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6717bcae-cb41-44b8-92e2-40b0d1ec95b4 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:20:38.229402 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6717bcae-cb41-44b8-92e2-40b0d1ec95b4 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:20:38.229674 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6717bcae-cb41-44b8-92e2-40b0d1ec95b4 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:20:38.231030 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 694/1391] 10.222.0.16 () {54 vars in 967 bytes} [Wed Aug 24 09:20:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:20:38.236029 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6a3e4a-758e-47f2-a2c5-69195ba4dbbd None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:20:38.236317 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6a3e4a-758e-47f2-a2c5-69195ba4dbbd None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:20:38.236589 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6a3e4a-758e-47f2-a2c5-69195ba4dbbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:20:38.254622 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:20:38.275079 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902aa1a3-db90-4c2f-bc0f-032bd89750e2 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:20:38.275363 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902aa1a3-db90-4c2f-bc0f-032bd89750e2 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:20:38.275669 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902aa1a3-db90-4c2f-bc0f-032bd89750e2 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:20:38.277020 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 695/1392] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:20:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 09:20:38.282572 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e81905-7047-4091-b3ab-974cc43959d9 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:20:38.282856 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e81905-7047-4091-b3ab-974cc43959d9 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:20:38.283134 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e81905-7047-4091-b3ab-974cc43959d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:20:38.318266 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-ef6a3e4a-758e-47f2-a2c5-69195ba4dbbd None None] MFA Rules not processed for user `ea2666d430a646d386104addf1f42888`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:20:38.319476 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-ef6a3e4a-758e-47f2-a2c5-69195ba4dbbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:20:38.361698 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-38e81905-7047-4091-b3ab-974cc43959d9 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:20:38.362915 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-38e81905-7047-4091-b3ab-974cc43959d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:20:38.403786 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 698/1393] 10.222.0.16 () {58 vars in 1064 bytes} [Wed Aug 24 09:20:38 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:20:38.411186 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6a3e4a-758e-47f2-a2c5-69195ba4dbbd None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:20:38.418747 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6a3e4a-758e-47f2-a2c5-69195ba4dbbd None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:20:38.464187 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 696/1394] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:20:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 09:20:38.495376 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0624921-abdb-412c-906e-84e8050b1889 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea2666d430a646d386104addf1f42888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:20:38.496611 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0624921-abdb-412c-906e-84e8050b1889 service glance] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:20:38.496894 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0624921-abdb-412c-906e-84e8050b1889 service glance] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:20:38.497170 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0624921-abdb-412c-906e-84e8050b1889 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:20:38.498814 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-d0624921-abdb-412c-906e-84e8050b1889 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:20:38.529452 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:20:38.609462 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0624921-abdb-412c-906e-84e8050b1889 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:20:38.610888 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0624921-abdb-412c-906e-84e8050b1889 service glance] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:20:38.629805 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 699/1395] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:20:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 09:20:38.759123 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e81905-7047-4091-b3ab-974cc43959d9 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:20:38.761336 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-38e81905-7047-4091-b3ab-974cc43959d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:20:38.774985 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:20:38.887219 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e81905-7047-4091-b3ab-974cc43959d9 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:20:39.005096 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c48b543-04bc-4572-a355-3d86117058f3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f055f7dc51d43bda9b315ca67f11d58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:20:39.005624 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c48b543-04bc-4572-a355-3d86117058f3 service nova] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:20:39.005900 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c48b543-04bc-4572-a355-3d86117058f3 service nova] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:20:39.006169 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c48b543-04bc-4572-a355-3d86117058f3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:20:39.007795 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-6c48b543-04bc-4572-a355-3d86117058f3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:20:39.016896 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:20:39.102535 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c48b543-04bc-4572-a355-3d86117058f3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:20:39.104223 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c48b543-04bc-4572-a355-3d86117058f3 service nova] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:20:39.123286 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 697/1396] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:20:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:20:39.530789 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6502853e-6ec8-4382-a384-83d47cb20139 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:20:39.531270 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6502853e-6ec8-4382-a384-83d47cb20139 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:20:39.531371 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6502853e-6ec8-4382-a384-83d47cb20139 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:20:39.598480 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-6502853e-6ec8-4382-a384-83d47cb20139 None None] MFA Rules not processed for user `0489462287fc4aea8e8e720af24e1ef8`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:20:39.599703 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-6502853e-6ec8-4382-a384-83d47cb20139 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:20:39.690560 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:20:39.696579 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 700/1397] 10.222.0.16 () {58 vars in 1009 bytes} [Wed Aug 24 09:20:39 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:20:39.744564 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6949f840-36e8-407c-a2e4-42d8c96d387a None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:20:39.744966 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6949f840-36e8-407c-a2e4-42d8c96d387a None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:20:39.745314 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6949f840-36e8-407c-a2e4-42d8c96d387a None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:20:39.747895 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 698/1398] 10.222.0.16 () {54 vars in 943 bytes} [Wed Aug 24 09:20:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:20:39.754855 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ce8a38-ccb8-474e-a707-9ed1000178f3 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:20:39.755369 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ce8a38-ccb8-474e-a707-9ed1000178f3 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:20:39.755684 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ce8a38-ccb8-474e-a707-9ed1000178f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:20:39.837606 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-05ce8a38-ccb8-474e-a707-9ed1000178f3 None None] MFA Rules not processed for user `fdd5f0d67fe54e9fae6f4af644988d93`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:20:39.838854 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-05ce8a38-ccb8-474e-a707-9ed1000178f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:20:39.950902 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 701/1399] 10.222.0.16 () {58 vars in 1040 bytes} [Wed Aug 24 09:20:39 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:20:39.960318 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6949f840-36e8-407c-a2e4-42d8c96d387a None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:20:39.961857 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-6949f840-36e8-407c-a2e4-42d8c96d387a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:20:40.099324 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:20:40.101197 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6949f840-36e8-407c-a2e4-42d8c96d387a None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:20:40.207339 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd95bec9-92da-478e-8296-e6856d6bb485 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdd5f0d67fe54e9fae6f4af644988d93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:20:40.207993 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd95bec9-92da-478e-8296-e6856d6bb485 service swift] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:20:40.208276 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd95bec9-92da-478e-8296-e6856d6bb485 service swift] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:20:40.208548 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd95bec9-92da-478e-8296-e6856d6bb485 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:20:40.210251 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-dd95bec9-92da-478e-8296-e6856d6bb485 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:20:40.296152 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd95bec9-92da-478e-8296-e6856d6bb485 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:20:40.297798 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd95bec9-92da-478e-8296-e6856d6bb485 service swift] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:20:40.307311 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 699/1400] 10.222.0.16 () {60 vars in 1384 bytes} [Wed Aug 24 09:20:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 09:20:41.776199 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed11fa23-3dcd-488a-8f79-3e5f784f25fd None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:20:41.776729 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed11fa23-3dcd-488a-8f79-3e5f784f25fd None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:20:41.776881 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed11fa23-3dcd-488a-8f79-3e5f784f25fd None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:20:41.779576 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 702/1401] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:20:41 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:20:41.786560 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82df1e85-87d2-480b-99cd-3072f596d29c None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:20:41.787052 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82df1e85-87d2-480b-99cd-3072f596d29c None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:20:41.787334 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82df1e85-87d2-480b-99cd-3072f596d29c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:20:41.849049 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:20:41.883706 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-82df1e85-87d2-480b-99cd-3072f596d29c None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:20:41.885359 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-82df1e85-87d2-480b-99cd-3072f596d29c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:20:42.024869 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 700/1402] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:20:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:20:42.333247 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed11fa23-3dcd-488a-8f79-3e5f784f25fd None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:20:42.334791 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-ed11fa23-3dcd-488a-8f79-3e5f784f25fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:20:42.405587 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:20:42.508738 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed11fa23-3dcd-488a-8f79-3e5f784f25fd None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:20:42.636147 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e592ce55-d26b-40cf-b2e8-be53f04acfec service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f055f7dc51d43bda9b315ca67f11d58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:20:42.637301 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e592ce55-d26b-40cf-b2e8-be53f04acfec service nova] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:20:42.637578 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e592ce55-d26b-40cf-b2e8-be53f04acfec service nova] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:20:42.637844 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e592ce55-d26b-40cf-b2e8-be53f04acfec service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:20:42.639946 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-e592ce55-d26b-40cf-b2e8-be53f04acfec service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:20:42.657725 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:20:42.743294 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e592ce55-d26b-40cf-b2e8-be53f04acfec service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:20:42.745020 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e592ce55-d26b-40cf-b2e8-be53f04acfec service nova] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:20:42.765799 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 703/1403] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:20:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:20:44.872576 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c25f37-94b4-4932-b2d1-02ddb1a7efd1 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:20:44.873164 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c25f37-94b4-4932-b2d1-02ddb1a7efd1 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:20:44.873164 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c25f37-94b4-4932-b2d1-02ddb1a7efd1 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:20:44.874631 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 701/1404] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:20:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:20:44.880605 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a24d7ba-cc2c-4e97-896b-f16ac16f1e0d None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:20:44.880605 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a24d7ba-cc2c-4e97-896b-f16ac16f1e0d None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:20:44.880605 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a24d7ba-cc2c-4e97-896b-f16ac16f1e0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:20:44.953058 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-4a24d7ba-cc2c-4e97-896b-f16ac16f1e0d None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:20:44.954284 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-4a24d7ba-cc2c-4e97-896b-f16ac16f1e0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:20:45.052788 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:20:45.060734 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 704/1405] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:20:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:20:45.352883 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c25f37-94b4-4932-b2d1-02ddb1a7efd1 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:20:45.354448 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-23c25f37-94b4-4932-b2d1-02ddb1a7efd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:20:45.473287 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:20:45.519880 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c25f37-94b4-4932-b2d1-02ddb1a7efd1 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:20:45.650118 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b823cfd-c5ae-4f8f-81a6-3dea83792da0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f055f7dc51d43bda9b315ca67f11d58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:20:45.650776 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b823cfd-c5ae-4f8f-81a6-3dea83792da0 service nova] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:20:45.651123 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b823cfd-c5ae-4f8f-81a6-3dea83792da0 service nova] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:20:45.651467 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b823cfd-c5ae-4f8f-81a6-3dea83792da0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:20:45.653755 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-8b823cfd-c5ae-4f8f-81a6-3dea83792da0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:20:45.736124 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:20:45.754440 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b823cfd-c5ae-4f8f-81a6-3dea83792da0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:20:45.756178 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b823cfd-c5ae-4f8f-81a6-3dea83792da0 service nova] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:20:45.776182 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 702/1406] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:20:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:20:47.876832 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4347c8fe-e6cb-469b-aaf1-20dbd8f957ac None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:20:47.877441 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4347c8fe-e6cb-469b-aaf1-20dbd8f957ac None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:20:47.877441 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4347c8fe-e6cb-469b-aaf1-20dbd8f957ac None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:20:47.882648 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 705/1407] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:20:47 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:20:47.884132 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124c6b1a-4cc0-4bba-bf47-5afa2d455ba8 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:20:47.884590 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124c6b1a-4cc0-4bba-bf47-5afa2d455ba8 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:20:47.884681 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124c6b1a-4cc0-4bba-bf47-5afa2d455ba8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:20:47.957807 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-124c6b1a-4cc0-4bba-bf47-5afa2d455ba8 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:20:47.959041 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-124c6b1a-4cc0-4bba-bf47-5afa2d455ba8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:20:48.063434 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 703/1408] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:20:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 09:20:48.359426 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4347c8fe-e6cb-469b-aaf1-20dbd8f957ac None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:20:48.361096 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-4347c8fe-e6cb-469b-aaf1-20dbd8f957ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:20:48.538225 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4347c8fe-e6cb-469b-aaf1-20dbd8f957ac None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:20:48.618916 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:20:48.671897 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa61f96f-7222-407a-817f-4d4e8ac8226f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f055f7dc51d43bda9b315ca67f11d58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:20:48.672507 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa61f96f-7222-407a-817f-4d4e8ac8226f service nova] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:20:48.672794 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa61f96f-7222-407a-817f-4d4e8ac8226f service nova] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:20:48.673066 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa61f96f-7222-407a-817f-4d4e8ac8226f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:20:48.674729 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-fa61f96f-7222-407a-817f-4d4e8ac8226f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:20:48.813899 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa61f96f-7222-407a-817f-4d4e8ac8226f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:20:48.815543 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa61f96f-7222-407a-817f-4d4e8ac8226f service nova] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:20:48.840991 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 706/1409] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:20:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 482 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 09:20:58.061525 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=78 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:20:58.837973 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=65 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:21:05.113039 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086d7766-c553-4b02-9157-af33e370cf87 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:21:05.113613 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086d7766-c553-4b02-9157-af33e370cf87 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:21:05.113613 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086d7766-c553-4b02-9157-af33e370cf87 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:21:05.114935 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 704/1410] 10.222.0.16 () {54 vars in 918 bytes} [Wed Aug 24 09:21:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:21:05.119985 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4456ca-f9a0-4480-a83c-97ab45a69dab None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:21:05.120410 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4456ca-f9a0-4480-a83c-97ab45a69dab None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:21:05.120520 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4456ca-f9a0-4480-a83c-97ab45a69dab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:21:05.195196 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-3f4456ca-f9a0-4480-a83c-97ab45a69dab None None] MFA Rules not processed for user `80cada22039d4f11935412ee578acd05`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:21:05.196427 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-3f4456ca-f9a0-4480-a83c-97ab45a69dab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:21:05.304719 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 707/1411] 10.222.0.16 () {58 vars in 1015 bytes} [Wed Aug 24 09:21:05 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:21:05.319316 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dae68a-3e42-4d6b-bab7-7dd6e1e7d6ef None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:21:05.320576 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dae68a-3e42-4d6b-bab7-7dd6e1e7d6ef None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:21:05.320576 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dae68a-3e42-4d6b-bab7-7dd6e1e7d6ef None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:21:05.321712 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 705/1412] 10.222.0.16 () {54 vars in 918 bytes} [Wed Aug 24 09:21:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:21:05.326804 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9a2f14-1a95-4f79-bcd8-d478f7070c3f None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:21:05.327090 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9a2f14-1a95-4f79-bcd8-d478f7070c3f None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:21:05.327415 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9a2f14-1a95-4f79-bcd8-d478f7070c3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:21:05.398638 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:21:05.401760 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-9a9a2f14-1a95-4f79-bcd8-d478f7070c3f None None] MFA Rules not processed for user `1f055f7dc51d43bda9b315ca67f11d58`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:21:05.402970 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-9a9a2f14-1a95-4f79-bcd8-d478f7070c3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:21:05.520132 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 708/1413] 10.222.0.16 () {58 vars in 1015 bytes} [Wed Aug 24 09:21:05 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:21:07.935544 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4542e2-3107-4c49-ba69-2598d1837ce7 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:21:07.936109 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4542e2-3107-4c49-ba69-2598d1837ce7 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:21:07.936109 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4542e2-3107-4c49-ba69-2598d1837ce7 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:21:07.937459 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 706/1414] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 09:21:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:21:07.942828 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f86f28f-1f76-40df-9efc-4eec3b7db0b0 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:21:07.943282 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f86f28f-1f76-40df-9efc-4eec3b7db0b0 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:21:07.943388 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f86f28f-1f76-40df-9efc-4eec3b7db0b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:21:08.011277 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-5f86f28f-1f76-40df-9efc-4eec3b7db0b0 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:21:08.012529 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-5f86f28f-1f76-40df-9efc-4eec3b7db0b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:21:08.067085 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:21:08.110458 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 709/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 09:21:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:21:08.397068 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4542e2-3107-4c49-ba69-2598d1837ce7 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:21:08.398637 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-6d4542e2-3107-4c49-ba69-2598d1837ce7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:21:08.545227 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4542e2-3107-4c49-ba69-2598d1837ce7 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:21:08.623914 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:21:08.665142 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53ed079-6c09-4934-b81c-d3c4c4332fff service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f055f7dc51d43bda9b315ca67f11d58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:21:08.665754 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53ed079-6c09-4934-b81c-d3c4c4332fff service nova] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:21:08.666038 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53ed079-6c09-4934-b81c-d3c4c4332fff service nova] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:21:08.666309 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53ed079-6c09-4934-b81c-d3c4c4332fff service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:21:08.668470 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-f53ed079-6c09-4934-b81c-d3c4c4332fff service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:21:08.757161 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53ed079-6c09-4934-b81c-d3c4c4332fff service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:21:08.758760 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53ed079-6c09-4934-b81c-d3c4c4332fff service nova] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:21:08.776672 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 707/1416] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:21:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:21:09.677549 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ddc5b2-60d0-484d-9148-0c380629da64 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:21:09.678069 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ddc5b2-60d0-484d-9148-0c380629da64 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:21:09.678069 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ddc5b2-60d0-484d-9148-0c380629da64 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:21:09.679341 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 710/1417] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 09:21:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 09:21:09.684391 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc900871-76a8-4791-891b-9c02f0e5e95d None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:21:09.684839 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc900871-76a8-4791-891b-9c02f0e5e95d None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:21:09.684965 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc900871-76a8-4791-891b-9c02f0e5e95d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:21:09.761063 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-cc900871-76a8-4791-891b-9c02f0e5e95d None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:21:09.762258 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-cc900871-76a8-4791-891b-9c02f0e5e95d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:21:09.788429 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:21:09.859198 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 708/1418] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 09:21:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:21:10.008031 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ddc5b2-60d0-484d-9148-0c380629da64 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:21:10.009485 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-65ddc5b2-60d0-484d-9148-0c380629da64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:21:10.156264 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ddc5b2-60d0-484d-9148-0c380629da64 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:21:10.195123 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:21:10.266716 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186863a4-1326-4c9a-9111-fc6773cfeca2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f055f7dc51d43bda9b315ca67f11d58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:21:10.267248 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186863a4-1326-4c9a-9111-fc6773cfeca2 service nova] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:21:10.267526 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186863a4-1326-4c9a-9111-fc6773cfeca2 service nova] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:21:10.267827 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186863a4-1326-4c9a-9111-fc6773cfeca2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:21:10.269466 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-186863a4-1326-4c9a-9111-fc6773cfeca2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:21:10.361330 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186863a4-1326-4c9a-9111-fc6773cfeca2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:21:10.362948 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186863a4-1326-4c9a-9111-fc6773cfeca2 service nova] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:21:10.381220 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 711/1419] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:21:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:21:11.262179 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a3f60c-715d-470d-8eb4-2a429ddd327c None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:21:11.262714 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a3f60c-715d-470d-8eb4-2a429ddd327c None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:21:11.262714 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a3f60c-715d-470d-8eb4-2a429ddd327c None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:21:11.264136 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 709/1420] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 09:21:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:21:11.269322 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf7a7b3-63d6-48fb-886a-b2f4887c21f7 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:21:11.269769 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf7a7b3-63d6-48fb-886a-b2f4887c21f7 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:21:11.269895 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf7a7b3-63d6-48fb-886a-b2f4887c21f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:21:11.328130 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:21:11.351405 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-4cf7a7b3-63d6-48fb-886a-b2f4887c21f7 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:21:11.352634 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-4cf7a7b3-63d6-48fb-886a-b2f4887c21f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:21:11.448482 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 712/1421] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 09:21:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:21:11.603169 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a3f60c-715d-470d-8eb4-2a429ddd327c None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:21:11.604741 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-b3a3f60c-715d-470d-8eb4-2a429ddd327c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:21:11.750358 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a3f60c-715d-470d-8eb4-2a429ddd327c None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:21:11.762639 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:21:11.861316 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585299b5-1c6c-45db-bf08-5839da6af95b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f055f7dc51d43bda9b315ca67f11d58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:21:11.861834 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585299b5-1c6c-45db-bf08-5839da6af95b service nova] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:21:11.862108 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585299b5-1c6c-45db-bf08-5839da6af95b service nova] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:21:11.862373 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585299b5-1c6c-45db-bf08-5839da6af95b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:21:11.863995 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-585299b5-1c6c-45db-bf08-5839da6af95b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:21:11.951127 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585299b5-1c6c-45db-bf08-5839da6af95b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:21:11.952745 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585299b5-1c6c-45db-bf08-5839da6af95b service nova] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:21:11.970567 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 710/1422] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:21:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:21:12.830063 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0caf27-4ecf-4a4c-b35e-4b0eac7c72ad None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:21:12.830625 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0caf27-4ecf-4a4c-b35e-4b0eac7c72ad None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:21:12.830726 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0caf27-4ecf-4a4c-b35e-4b0eac7c72ad None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:21:12.832030 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 713/1423] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 09:21:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:21:12.837223 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ecc414-2e18-4547-bc0a-647476e64f04 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:21:12.837918 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ecc414-2e18-4547-bc0a-647476e64f04 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:21:12.838031 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ecc414-2e18-4547-bc0a-647476e64f04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:21:12.858796 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:21:12.914476 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-50ecc414-2e18-4547-bc0a-647476e64f04 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:21:12.915767 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-50ecc414-2e18-4547-bc0a-647476e64f04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:21:13.011319 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 711/1424] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 09:21:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:21:13.161733 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0caf27-4ecf-4a4c-b35e-4b0eac7c72ad None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:21:13.163328 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-9c0caf27-4ecf-4a4c-b35e-4b0eac7c72ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:21:13.273472 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:21:13.309422 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0caf27-4ecf-4a4c-b35e-4b0eac7c72ad None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:21:13.424935 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b79d8dc-b551-42e9-b60c-044bbb7bcd06 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f055f7dc51d43bda9b315ca67f11d58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:21:13.425505 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b79d8dc-b551-42e9-b60c-044bbb7bcd06 service nova] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:21:13.425781 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b79d8dc-b551-42e9-b60c-044bbb7bcd06 service nova] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:21:13.426047 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b79d8dc-b551-42e9-b60c-044bbb7bcd06 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:21:13.427725 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-7b79d8dc-b551-42e9-b60c-044bbb7bcd06 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:21:13.503577 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:21:13.517514 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b79d8dc-b551-42e9-b60c-044bbb7bcd06 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:21:13.519117 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b79d8dc-b551-42e9-b60c-044bbb7bcd06 service nova] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:21:13.536990 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 714/1425] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:21:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:21:14.387258 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d057c0b-e594-4f53-8fad-6613a48d2f6c None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:21:14.387843 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d057c0b-e594-4f53-8fad-6613a48d2f6c None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:21:14.387843 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d057c0b-e594-4f53-8fad-6613a48d2f6c None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:21:14.389182 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 712/1426] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 09:21:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:21:14.394138 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9599ce1-913b-4c9e-86d0-469619ed6b86 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:21:14.394558 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9599ce1-913b-4c9e-86d0-469619ed6b86 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:21:14.394680 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9599ce1-913b-4c9e-86d0-469619ed6b86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:21:14.470099 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-d9599ce1-913b-4c9e-86d0-469619ed6b86 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:21:14.471377 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-d9599ce1-913b-4c9e-86d0-469619ed6b86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:21:14.568435 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 715/1427] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 09:21:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:21:14.718808 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d057c0b-e594-4f53-8fad-6613a48d2f6c None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:21:14.720403 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-7d057c0b-e594-4f53-8fad-6613a48d2f6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:21:14.804950 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:21:14.867739 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d057c0b-e594-4f53-8fad-6613a48d2f6c None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:21:14.979681 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a93441-c8d2-4e50-990b-7f0523d510d7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f055f7dc51d43bda9b315ca67f11d58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:21:14.980292 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a93441-c8d2-4e50-990b-7f0523d510d7 service nova] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:21:14.980575 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a93441-c8d2-4e50-990b-7f0523d510d7 service nova] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:21:14.980847 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a93441-c8d2-4e50-990b-7f0523d510d7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:21:14.982470 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-13a93441-c8d2-4e50-990b-7f0523d510d7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:21:15.031830 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:21:15.071176 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a93441-c8d2-4e50-990b-7f0523d510d7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:21:15.073411 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a93441-c8d2-4e50-990b-7f0523d510d7 service nova] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:21:15.091325 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 713/1428] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:21:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:21:15.993879 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c463183e-38b5-456f-84d0-f995f4732317 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:21:15.994565 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c463183e-38b5-456f-84d0-f995f4732317 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:21:15.994565 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c463183e-38b5-456f-84d0-f995f4732317 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:21:15.995828 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 716/1429] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 09:21:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:21:16.000618 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d40dea-7eac-4b9b-b267-bce5bde98281 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:21:16.001096 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d40dea-7eac-4b9b-b267-bce5bde98281 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:21:16.001206 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d40dea-7eac-4b9b-b267-bce5bde98281 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:21:16.075975 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-b5d40dea-7eac-4b9b-b267-bce5bde98281 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:21:16.077185 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-b5d40dea-7eac-4b9b-b267-bce5bde98281 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:21:16.171907 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 714/1430] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 09:21:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:21:16.321866 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c463183e-38b5-456f-84d0-f995f4732317 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:21:16.323962 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-c463183e-38b5-456f-84d0-f995f4732317 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:21:16.373068 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:21:16.470488 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c463183e-38b5-456f-84d0-f995f4732317 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:21:16.581640 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffbb5ab-22e3-4d7c-81c4-810231afd790 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f055f7dc51d43bda9b315ca67f11d58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:21:16.582253 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffbb5ab-22e3-4d7c-81c4-810231afd790 service nova] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:21:16.582532 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffbb5ab-22e3-4d7c-81c4-810231afd790 service nova] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:21:16.582801 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffbb5ab-22e3-4d7c-81c4-810231afd790 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:21:16.584443 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-4ffbb5ab-22e3-4d7c-81c4-810231afd790 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:21:16.592682 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:21:16.677329 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffbb5ab-22e3-4d7c-81c4-810231afd790 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:21:16.679247 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffbb5ab-22e3-4d7c-81c4-810231afd790 service nova] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:21:16.698973 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 717/1431] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:21:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:21:17.580120 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bdf2db-9edb-4ca4-9a30-e493753960a0 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:21:17.580659 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bdf2db-9edb-4ca4-9a30-e493753960a0 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:21:17.580659 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bdf2db-9edb-4ca4-9a30-e493753960a0 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:21:17.581988 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 715/1432] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 09:21:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:21:17.586988 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ca77d9-5be2-4a2d-a0b2-03f27b33fdab None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:21:17.587718 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ca77d9-5be2-4a2d-a0b2-03f27b33fdab None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:21:17.587875 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ca77d9-5be2-4a2d-a0b2-03f27b33fdab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:21:17.662654 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-48ca77d9-5be2-4a2d-a0b2-03f27b33fdab None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:21:17.663881 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-48ca77d9-5be2-4a2d-a0b2-03f27b33fdab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:21:17.732470 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:21:17.762329 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 718/1433] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 09:21:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:21:17.912620 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bdf2db-9edb-4ca4-9a30-e493753960a0 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:21:17.914066 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-71bdf2db-9edb-4ca4-9a30-e493753960a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:21:17.931957 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:21:18.060264 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bdf2db-9edb-4ca4-9a30-e493753960a0 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:21:18.150071 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:21:18.176085 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7c3bae-ae5a-471b-be33-177f5742901c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f055f7dc51d43bda9b315ca67f11d58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:21:18.176626 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7c3bae-ae5a-471b-be33-177f5742901c service nova] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:21:18.176929 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7c3bae-ae5a-471b-be33-177f5742901c service nova] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:21:18.177200 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7c3bae-ae5a-471b-be33-177f5742901c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:21:18.179349 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-fa7c3bae-ae5a-471b-be33-177f5742901c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:21:18.267602 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7c3bae-ae5a-471b-be33-177f5742901c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:21:18.269213 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7c3bae-ae5a-471b-be33-177f5742901c service nova] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:21:18.287015 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 716/1434] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:21:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:21:19.151566 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11d4c7d-76df-4922-a26b-bc3658bcb227 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:21:19.152158 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11d4c7d-76df-4922-a26b-bc3658bcb227 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:21:19.152158 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11d4c7d-76df-4922-a26b-bc3658bcb227 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:21:19.154477 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 719/1435] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 09:21:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:21:19.159413 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5a81d8-3553-462e-99eb-243fec2b6df7 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:21:19.160033 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5a81d8-3553-462e-99eb-243fec2b6df7 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:21:19.160033 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5a81d8-3553-462e-99eb-243fec2b6df7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:21:19.234776 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-2b5a81d8-3553-462e-99eb-243fec2b6df7 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:21:19.236007 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-2b5a81d8-3553-462e-99eb-243fec2b6df7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:21:19.276871 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:21:19.331632 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 717/1436] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 09:21:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:21:19.493951 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11d4c7d-76df-4922-a26b-bc3658bcb227 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:21:19.495446 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-f11d4c7d-76df-4922-a26b-bc3658bcb227 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:21:19.640047 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11d4c7d-76df-4922-a26b-bc3658bcb227 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:21:19.692262 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:21:19.755261 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e513a994-c20b-449a-a294-753b69af2491 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f055f7dc51d43bda9b315ca67f11d58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:21:19.755950 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e513a994-c20b-449a-a294-753b69af2491 service nova] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:21:19.756226 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e513a994-c20b-449a-a294-753b69af2491 service nova] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:21:19.756497 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e513a994-c20b-449a-a294-753b69af2491 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:21:19.758442 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-e513a994-c20b-449a-a294-753b69af2491 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:21:19.852381 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e513a994-c20b-449a-a294-753b69af2491 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:21:19.854005 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e513a994-c20b-449a-a294-753b69af2491 service nova] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:21:19.872506 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 720/1437] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:21:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:21:20.792368 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3c98a4-07cc-4374-95cd-93a27917a4f8 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:21:20.792920 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3c98a4-07cc-4374-95cd-93a27917a4f8 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:21:20.792920 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3c98a4-07cc-4374-95cd-93a27917a4f8 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:21:20.794268 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 718/1438] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 09:21:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:21:20.799157 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c61db4-6f6f-42bf-baa9-f92ecee6ee90 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:21:20.799561 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c61db4-6f6f-42bf-baa9-f92ecee6ee90 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:21:20.799734 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c61db4-6f6f-42bf-baa9-f92ecee6ee90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:21:20.878791 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-32c61db4-6f6f-42bf-baa9-f92ecee6ee90 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:21:20.880106 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-32c61db4-6f6f-42bf-baa9-f92ecee6ee90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:21:20.882212 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:21:20.976959 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 721/1439] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 09:21:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:21:21.129715 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3c98a4-07cc-4374-95cd-93a27917a4f8 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:21:21.131184 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-da3c98a4-07cc-4374-95cd-93a27917a4f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:21:21.274537 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3c98a4-07cc-4374-95cd-93a27917a4f8 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:21:21.301990 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:21:21.389461 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccea4131-adc3-4932-bf5c-18c4d69c9ed6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f055f7dc51d43bda9b315ca67f11d58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:21:21.389996 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccea4131-adc3-4932-bf5c-18c4d69c9ed6 service nova] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:21:21.390277 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccea4131-adc3-4932-bf5c-18c4d69c9ed6 service nova] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:21:21.390546 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccea4131-adc3-4932-bf5c-18c4d69c9ed6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:21:21.392267 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-ccea4131-adc3-4932-bf5c-18c4d69c9ed6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:21:21.482941 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccea4131-adc3-4932-bf5c-18c4d69c9ed6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:21:21.484565 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccea4131-adc3-4932-bf5c-18c4d69c9ed6 service nova] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:21:21.502888 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 719/1440] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:21:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:21:23.311081 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54eba198-f577-4a3f-819d-e7e68f7ce978 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:21:23.311558 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54eba198-f577-4a3f-819d-e7e68f7ce978 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:21:23.311724 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54eba198-f577-4a3f-819d-e7e68f7ce978 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:21:23.313197 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 722/1441] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 09:21:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:21:23.318564 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d2d746-268f-46a9-90de-3abb2f154e55 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:21:23.318998 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d2d746-268f-46a9-90de-3abb2f154e55 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:21:23.319102 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d2d746-268f-46a9-90de-3abb2f154e55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:21:23.361422 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:21:23.386983 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-13d2d746-268f-46a9-90de-3abb2f154e55 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:21:23.388187 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-13d2d746-268f-46a9-90de-3abb2f154e55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:21:23.483311 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 720/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 09:21:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:21:23.766418 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54eba198-f577-4a3f-819d-e7e68f7ce978 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:21:23.768041 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-54eba198-f577-4a3f-819d-e7e68f7ce978 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:21:23.872678 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54eba198-f577-4a3f-819d-e7e68f7ce978 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:21:23.904746 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:21:23.946322 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c37ff1-9d10-4444-bf84-516626ed3195 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e5260e798404e8bb12b25ca8ea97052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:21:23.946968 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c37ff1-9d10-4444-bf84-516626ed3195 service neutron] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:21:23.947277 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c37ff1-9d10-4444-bf84-516626ed3195 service neutron] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:21:23.947554 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c37ff1-9d10-4444-bf84-516626ed3195 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:21:23.949841 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-e8c37ff1-9d10-4444-bf84-516626ed3195 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:21:24.057288 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c37ff1-9d10-4444-bf84-516626ed3195 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:21:24.058742 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c37ff1-9d10-4444-bf84-516626ed3195 service neutron] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:21:24.076993 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 723/1443] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:21:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:21:26.134212 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b72db0-4e0e-488f-9527-47069c1a0b7c None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:21:26.134780 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b72db0-4e0e-488f-9527-47069c1a0b7c None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:21:26.134780 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b72db0-4e0e-488f-9527-47069c1a0b7c None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:21:26.136335 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 721/1444] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 09:21:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:21:26.142033 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b34c6d-dff9-4635-93ba-d6913517fb7f None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:21:26.142518 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b34c6d-dff9-4635-93ba-d6913517fb7f None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:21:26.142635 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b34c6d-dff9-4635-93ba-d6913517fb7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:21:26.207636 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-b6b34c6d-dff9-4635-93ba-d6913517fb7f None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:21:26.208900 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-b6b34c6d-dff9-4635-93ba-d6913517fb7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:21:26.221896 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:21:26.306555 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 724/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 09:21:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 09:21:26.588834 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b72db0-4e0e-488f-9527-47069c1a0b7c None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:21:26.590956 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-71b72db0-4e0e-488f-9527-47069c1a0b7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:21:26.691690 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b72db0-4e0e-488f-9527-47069c1a0b7c None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:21:26.703428 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:21:26.763983 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d2337e-2036-42b7-ba16-00972c4db4ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e5260e798404e8bb12b25ca8ea97052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:21:26.764506 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d2337e-2036-42b7-ba16-00972c4db4ed service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:21:26.764783 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d2337e-2036-42b7-ba16-00972c4db4ed service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:21:26.765051 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d2337e-2036-42b7-ba16-00972c4db4ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:21:26.766652 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-d4d2337e-2036-42b7-ba16-00972c4db4ed service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:21:26.878564 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d2337e-2036-42b7-ba16-00972c4db4ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:21:26.880073 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d2337e-2036-42b7-ba16-00972c4db4ed service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:21:26.898254 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 722/1446] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:21:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:21:29.211310 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6673561a-649b-4730-86b0-acf161a345a6 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:21:29.211936 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6673561a-649b-4730-86b0-acf161a345a6 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:21:29.211936 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6673561a-649b-4730-86b0-acf161a345a6 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:21:29.213477 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 725/1447] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 09:21:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 09:21:29.219406 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fdc334-92e5-41e8-a3cb-340a50c11a6c None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:21:29.219839 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fdc334-92e5-41e8-a3cb-340a50c11a6c None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:21:29.219998 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fdc334-92e5-41e8-a3cb-340a50c11a6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:21:29.268020 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:21:29.285741 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-41fdc334-92e5-41e8-a3cb-340a50c11a6c None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:21:29.286953 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-41fdc334-92e5-41e8-a3cb-340a50c11a6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:21:29.381993 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 723/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 09:21:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 09:21:29.663679 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6673561a-649b-4730-86b0-acf161a345a6 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:21:29.665243 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-6673561a-649b-4730-86b0-acf161a345a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:21:29.768999 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6673561a-649b-4730-86b0-acf161a345a6 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:21:29.810258 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:21:29.843221 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5429bc63-8421-4b7b-ad73-2bd0fb0805ac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e5260e798404e8bb12b25ca8ea97052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:21:29.843872 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5429bc63-8421-4b7b-ad73-2bd0fb0805ac service neutron] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:21:29.844156 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5429bc63-8421-4b7b-ad73-2bd0fb0805ac service neutron] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:21:29.844432 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5429bc63-8421-4b7b-ad73-2bd0fb0805ac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:21:29.846059 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-5429bc63-8421-4b7b-ad73-2bd0fb0805ac service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:21:29.957140 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5429bc63-8421-4b7b-ad73-2bd0fb0805ac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:21:29.958492 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5429bc63-8421-4b7b-ad73-2bd0fb0805ac service neutron] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:21:29.976469 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 726/1449] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:21:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:21:32.014124 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8ebef2-a514-490f-a522-9148632590a1 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:21:32.014704 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8ebef2-a514-490f-a522-9148632590a1 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:21:32.014704 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8ebef2-a514-490f-a522-9148632590a1 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:21:32.016064 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 724/1450] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 09:21:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:21:32.021587 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e58d44-203c-439c-be73-1ec9582f8813 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:21:32.022073 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e58d44-203c-439c-be73-1ec9582f8813 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:21:32.022182 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e58d44-203c-439c-be73-1ec9582f8813 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:21:32.091216 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-01e58d44-203c-439c-be73-1ec9582f8813 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:21:32.092514 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-01e58d44-203c-439c-be73-1ec9582f8813 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:21:32.112860 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:21:32.188121 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 727/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 09:21:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:21:32.475080 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8ebef2-a514-490f-a522-9148632590a1 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:21:32.476683 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-df8ebef2-a514-490f-a522-9148632590a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:21:32.580961 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8ebef2-a514-490f-a522-9148632590a1 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:21:32.600355 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:21:32.653883 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb77d4b-a6d3-45a8-b049-6cbc0e5cab22 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e5260e798404e8bb12b25ca8ea97052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:21:32.654482 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb77d4b-a6d3-45a8-b049-6cbc0e5cab22 service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:21:32.654756 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb77d4b-a6d3-45a8-b049-6cbc0e5cab22 service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:21:32.655047 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb77d4b-a6d3-45a8-b049-6cbc0e5cab22 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:21:32.656671 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-dbb77d4b-a6d3-45a8-b049-6cbc0e5cab22 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:21:32.762717 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb77d4b-a6d3-45a8-b049-6cbc0e5cab22 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:21:32.764212 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb77d4b-a6d3-45a8-b049-6cbc0e5cab22 service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:21:32.782197 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 725/1452] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:21:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:21:34.762998 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fa9d4f-2c0c-4877-bf3d-51d191a58a3d None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:21:34.763538 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fa9d4f-2c0c-4877-bf3d-51d191a58a3d None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:21:34.763538 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fa9d4f-2c0c-4877-bf3d-51d191a58a3d None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:21:34.764945 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 728/1453] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 09:21:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:21:34.770570 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bc24ba-dada-44c9-80f1-560914343192 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:21:34.770965 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bc24ba-dada-44c9-80f1-560914343192 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:21:34.771124 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bc24ba-dada-44c9-80f1-560914343192 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:21:34.834958 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:21:34.837856 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-e6bc24ba-dada-44c9-80f1-560914343192 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:21:34.839097 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-e6bc24ba-dada-44c9-80f1-560914343192 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:21:34.934149 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 726/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 09:21:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:21:35.213863 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fa9d4f-2c0c-4877-bf3d-51d191a58a3d None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:21:35.215989 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-a0fa9d4f-2c0c-4877-bf3d-51d191a58a3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:21:35.319777 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fa9d4f-2c0c-4877-bf3d-51d191a58a3d None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:21:35.371274 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:21:35.396140 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a53587b-b4e1-4f54-8e2f-94e879c5f744 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e5260e798404e8bb12b25ca8ea97052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:21:35.396752 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a53587b-b4e1-4f54-8e2f-94e879c5f744 service neutron] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:21:35.397031 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a53587b-b4e1-4f54-8e2f-94e879c5f744 service neutron] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:21:35.397303 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a53587b-b4e1-4f54-8e2f-94e879c5f744 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:21:35.398904 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-2a53587b-b4e1-4f54-8e2f-94e879c5f744 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:21:35.507076 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a53587b-b4e1-4f54-8e2f-94e879c5f744 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:21:35.508450 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a53587b-b4e1-4f54-8e2f-94e879c5f744 service neutron] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:21:35.526590 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 729/1455] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:21:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:21:37.480935 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64af1e43-ccc5-4c41-aaf2-036fea1ea6a2 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:21:37.481498 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64af1e43-ccc5-4c41-aaf2-036fea1ea6a2 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:21:37.481498 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64af1e43-ccc5-4c41-aaf2-036fea1ea6a2 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:21:37.482858 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 727/1456] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 09:21:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:21:37.488307 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e763cfd8-41dc-47c8-905d-75e7a22224da None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:21:37.488710 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e763cfd8-41dc-47c8-905d-75e7a22224da None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:21:37.488860 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e763cfd8-41dc-47c8-905d-75e7a22224da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:21:37.555684 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-e763cfd8-41dc-47c8-905d-75e7a22224da None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:21:37.556839 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-e763cfd8-41dc-47c8-905d-75e7a22224da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:21:37.585677 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:21:37.653294 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 730/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 09:21:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:21:37.933901 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64af1e43-ccc5-4c41-aaf2-036fea1ea6a2 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:21:37.935463 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-64af1e43-ccc5-4c41-aaf2-036fea1ea6a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:21:38.036692 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64af1e43-ccc5-4c41-aaf2-036fea1ea6a2 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:21:38.064717 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:21:38.110251 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2a665b-ba60-4b8c-badf-9487166b402f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e5260e798404e8bb12b25ca8ea97052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:21:38.110848 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2a665b-ba60-4b8c-badf-9487166b402f service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:21:38.111125 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2a665b-ba60-4b8c-badf-9487166b402f service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:21:38.111393 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2a665b-ba60-4b8c-badf-9487166b402f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:21:38.113007 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-dc2a665b-ba60-4b8c-badf-9487166b402f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:21:38.218761 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2a665b-ba60-4b8c-badf-9487166b402f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:21:38.220147 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2a665b-ba60-4b8c-badf-9487166b402f service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:21:38.238844 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 728/1458] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:21:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:21:40.194923 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23835e6f-22b6-4f3e-8a47-3af1ec6c32b1 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:21:40.195491 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23835e6f-22b6-4f3e-8a47-3af1ec6c32b1 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:21:40.195491 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23835e6f-22b6-4f3e-8a47-3af1ec6c32b1 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:21:40.196833 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 731/1459] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 09:21:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:21:40.202336 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbed4c6-1875-4ff5-873f-af8d6a71e098 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:21:40.202735 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbed4c6-1875-4ff5-873f-af8d6a71e098 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:21:40.202885 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbed4c6-1875-4ff5-873f-af8d6a71e098 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:21:40.272526 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-9bbed4c6-1875-4ff5-873f-af8d6a71e098 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:21:40.278783 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-9bbed4c6-1875-4ff5-873f-af8d6a71e098 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:21:40.282363 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:21:40.374174 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 729/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 09:21:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:21:40.658496 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23835e6f-22b6-4f3e-8a47-3af1ec6c32b1 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:21:40.660095 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-23835e6f-22b6-4f3e-8a47-3af1ec6c32b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:21:40.762001 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23835e6f-22b6-4f3e-8a47-3af1ec6c32b1 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:21:40.815250 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:21:40.834108 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e57eae-5572-462e-a475-ff0f437354a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e5260e798404e8bb12b25ca8ea97052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:21:40.834714 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e57eae-5572-462e-a475-ff0f437354a1 service neutron] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:21:40.835031 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e57eae-5572-462e-a475-ff0f437354a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:21:40.835304 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e57eae-5572-462e-a475-ff0f437354a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:21:40.836929 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-29e57eae-5572-462e-a475-ff0f437354a1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:21:40.947192 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e57eae-5572-462e-a475-ff0f437354a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:21:40.949457 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e57eae-5572-462e-a475-ff0f437354a1 service neutron] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:21:40.968948 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 732/1461] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:21:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:21:42.945919 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5decb1b2-68b1-4808-bd2c-f7399a7c2eee None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:21:42.946506 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5decb1b2-68b1-4808-bd2c-f7399a7c2eee None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:21:42.946506 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5decb1b2-68b1-4808-bd2c-f7399a7c2eee None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:21:42.947857 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 730/1462] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 09:21:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:21:42.953251 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcda1f8-ad8e-42fa-914c-3dfa6864a024 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:21:42.953699 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcda1f8-ad8e-42fa-914c-3dfa6864a024 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:21:42.953900 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcda1f8-ad8e-42fa-914c-3dfa6864a024 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:21:43.019835 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-0bcda1f8-ad8e-42fa-914c-3dfa6864a024 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:21:43.021043 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-0bcda1f8-ad8e-42fa-914c-3dfa6864a024 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:21:43.059263 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:21:43.116418 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 733/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 09:21:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:21:43.416844 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5decb1b2-68b1-4808-bd2c-f7399a7c2eee None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:21:43.418397 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-5decb1b2-68b1-4808-bd2c-f7399a7c2eee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:21:43.524065 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5decb1b2-68b1-4808-bd2c-f7399a7c2eee None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:21:43.561879 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:21:43.598141 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb127652-4441-453e-975a-6bb14361bd73 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e5260e798404e8bb12b25ca8ea97052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:21:43.598740 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb127652-4441-453e-975a-6bb14361bd73 service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:21:43.599014 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb127652-4441-453e-975a-6bb14361bd73 service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:21:43.599279 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb127652-4441-453e-975a-6bb14361bd73 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:21:43.600896 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-cb127652-4441-453e-975a-6bb14361bd73 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:21:43.709082 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb127652-4441-453e-975a-6bb14361bd73 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:21:43.710545 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb127652-4441-453e-975a-6bb14361bd73 service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:21:43.731150 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 731/1464] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:21:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:21:45.659335 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f2abbd-b68b-4c86-bab0-e551a06aceb2 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:21:45.659941 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f2abbd-b68b-4c86-bab0-e551a06aceb2 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:21:45.659941 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f2abbd-b68b-4c86-bab0-e551a06aceb2 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:21:45.661266 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 734/1465] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 09:21:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:21:45.666543 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10daa37b-8fbf-4dca-a62e-108eefc63a18 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:21:45.666940 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10daa37b-8fbf-4dca-a62e-108eefc63a18 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:21:45.667151 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10daa37b-8fbf-4dca-a62e-108eefc63a18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:21:45.732096 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-10daa37b-8fbf-4dca-a62e-108eefc63a18 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:21:45.733282 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-10daa37b-8fbf-4dca-a62e-108eefc63a18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:21:45.750962 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:21:45.829046 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 732/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 09:21:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:21:46.137642 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f2abbd-b68b-4c86-bab0-e551a06aceb2 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:21:46.139320 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-06f2abbd-b68b-4c86-bab0-e551a06aceb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:21:46.244882 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f2abbd-b68b-4c86-bab0-e551a06aceb2 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:21:46.315485 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:21:46.322233 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8bec71-50ab-41f6-a161-fb23c2d68d8f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e5260e798404e8bb12b25ca8ea97052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:21:46.323339 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8bec71-50ab-41f6-a161-fb23c2d68d8f service neutron] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:21:46.323667 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8bec71-50ab-41f6-a161-fb23c2d68d8f service neutron] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:21:46.323947 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8bec71-50ab-41f6-a161-fb23c2d68d8f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:21:46.325561 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-5c8bec71-50ab-41f6-a161-fb23c2d68d8f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:21:46.434390 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8bec71-50ab-41f6-a161-fb23c2d68d8f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:21:46.435783 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8bec71-50ab-41f6-a161-fb23c2d68d8f service neutron] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:21:46.454488 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 735/1467] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:21:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:21:48.368899 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e22ca1-c356-461d-9da4-78cda41dbd7b None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:21:48.369431 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e22ca1-c356-461d-9da4-78cda41dbd7b None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:21:48.369431 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e22ca1-c356-461d-9da4-78cda41dbd7b None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:21:48.371333 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 733/1468] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 09:21:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:21:48.376882 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edb72e1-482e-4305-a1b8-4af30d83f363 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:21:48.377321 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edb72e1-482e-4305-a1b8-4af30d83f363 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:21:48.377428 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edb72e1-482e-4305-a1b8-4af30d83f363 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:21:48.443496 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-7edb72e1-482e-4305-a1b8-4af30d83f363 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:21:48.444732 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-7edb72e1-482e-4305-a1b8-4af30d83f363 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:21:48.492653 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:21:48.541771 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 736/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 09:21:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:21:48.819770 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e22ca1-c356-461d-9da4-78cda41dbd7b None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:21:48.821326 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-e8e22ca1-c356-461d-9da4-78cda41dbd7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:21:48.922567 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e22ca1-c356-461d-9da4-78cda41dbd7b None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:21:48.970757 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:21:48.997150 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e308f7a-38d9-453d-aec7-3e6d3fe8c043 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e5260e798404e8bb12b25ca8ea97052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:21:48.997755 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e308f7a-38d9-453d-aec7-3e6d3fe8c043 service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:21:48.998031 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e308f7a-38d9-453d-aec7-3e6d3fe8c043 service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:21:48.998318 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e308f7a-38d9-453d-aec7-3e6d3fe8c043 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:21:49.000127 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-2e308f7a-38d9-453d-aec7-3e6d3fe8c043 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:21:49.114441 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e308f7a-38d9-453d-aec7-3e6d3fe8c043 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:21:49.115809 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e308f7a-38d9-453d-aec7-3e6d3fe8c043 service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:21:49.133776 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 734/1470] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:21:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:21:58.539842 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=22 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:21:59.131542 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=66 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:22:08.560916 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2b4436-9e8a-413c-a0ca-759d9febad0c None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:22:08.561465 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2b4436-9e8a-413c-a0ca-759d9febad0c None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:22:08.561465 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2b4436-9e8a-413c-a0ca-759d9febad0c None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:22:08.562775 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 737/1471] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:22:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:22:08.568028 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4925f6c1-61d5-4159-9c97-b479fc92e9dc None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:22:08.574442 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4925f6c1-61d5-4159-9c97-b479fc92e9dc None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:22:08.574442 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4925f6c1-61d5-4159-9c97-b479fc92e9dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:22:08.634302 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-4925f6c1-61d5-4159-9c97-b479fc92e9dc None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:22:08.635542 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-4925f6c1-61d5-4159-9c97-b479fc92e9dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:22:08.734656 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 735/1472] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:22:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:22:09.030258 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9765843-e6f6-44ef-ab6e-4ee0541a0a27 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:22:09.030544 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9765843-e6f6-44ef-ab6e-4ee0541a0a27 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:22:09.030819 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9765843-e6f6-44ef-ab6e-4ee0541a0a27 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:22:09.032238 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 738/1473] 10.222.0.16 () {54 vars in 967 bytes} [Wed Aug 24 09:22:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:22:09.037061 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4302b1f5-4202-4407-b9bb-ba36fbc3bd1b None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:22:09.037353 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4302b1f5-4202-4407-b9bb-ba36fbc3bd1b None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:22:09.037631 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4302b1f5-4202-4407-b9bb-ba36fbc3bd1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:22:09.112613 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-4302b1f5-4202-4407-b9bb-ba36fbc3bd1b None None] MFA Rules not processed for user `ea2666d430a646d386104addf1f42888`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:22:09.113814 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-4302b1f5-4202-4407-b9bb-ba36fbc3bd1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:22:09.133714 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:22:09.188975 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 736/1474] 10.222.0.16 () {58 vars in 1064 bytes} [Wed Aug 24 09:22:09 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:22:09.196468 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9765843-e6f6-44ef-ab6e-4ee0541a0a27 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:22:09.198646 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-c9765843-e6f6-44ef-ab6e-4ee0541a0a27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:22:09.301204 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9765843-e6f6-44ef-ab6e-4ee0541a0a27 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:22:09.375562 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21844f6b-58f3-46c7-9f28-8956065a6b76 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea2666d430a646d386104addf1f42888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:22:09.376190 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21844f6b-58f3-46c7-9f28-8956065a6b76 service glance] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:22:09.376467 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21844f6b-58f3-46c7-9f28-8956065a6b76 service glance] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:22:09.376756 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21844f6b-58f3-46c7-9f28-8956065a6b76 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:22:09.378391 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-21844f6b-58f3-46c7-9f28-8956065a6b76 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:22:09.431123 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:22:09.488221 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21844f6b-58f3-46c7-9f28-8956065a6b76 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:22:09.489680 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21844f6b-58f3-46c7-9f28-8956065a6b76 service glance] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:22:09.508080 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 739/1475] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:22:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:22:11.500691 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7dede3-bded-4559-9d66-282d62f27d69 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:22:11.501222 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7dede3-bded-4559-9d66-282d62f27d69 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:22:11.501222 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7dede3-bded-4559-9d66-282d62f27d69 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:22:11.502499 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 737/1476] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:22:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:22:11.507768 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e03f313-7a44-4a3c-abb5-d9dafd30df9e None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:22:11.508183 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e03f313-7a44-4a3c-abb5-d9dafd30df9e None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:22:11.508324 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e03f313-7a44-4a3c-abb5-d9dafd30df9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:22:11.578522 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-0e03f313-7a44-4a3c-abb5-d9dafd30df9e None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:22:11.579749 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-0e03f313-7a44-4a3c-abb5-d9dafd30df9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:22:11.675849 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 740/1477] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:22:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:22:11.950269 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7dede3-bded-4559-9d66-282d62f27d69 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:22:11.951926 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-5b7dede3-bded-4559-9d66-282d62f27d69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:22:12.077769 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7dede3-bded-4559-9d66-282d62f27d69 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:22:12.125817 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:22:12.170886 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13f093d-88cd-47cc-85ef-9a4d8744cece admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:22:12.171487 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13f093d-88cd-47cc-85ef-9a4d8744cece admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:22:12.171796 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13f093d-88cd-47cc-85ef-9a4d8744cece admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:22:12.172068 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13f093d-88cd-47cc-85ef-9a4d8744cece admin admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:22:12.172701 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13f093d-88cd-47cc-85ef-9a4d8744cece admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:22:12.174448 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: WARNING py.warnings [None req-d13f093d-88cd-47cc-85ef-9a4d8744cece admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 09:22:12.174448 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: warnings.warn(msg) Aug 24 09:22:12.174448 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: Aug 24 09:22:12.175621 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13f093d-88cd-47cc-85ef-9a4d8744cece admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:22:12.194679 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 738/1478] 10.222.0.16 () {56 vars in 1108 bytes} [Wed Aug 24 09:22:11 2022] GET /identity/v3/projects => generated 2553 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:22:14.138641 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92da538c-b307-4e78-856b-b03ec1f0e1ba None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:22:14.139235 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92da538c-b307-4e78-856b-b03ec1f0e1ba None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:22:14.139235 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92da538c-b307-4e78-856b-b03ec1f0e1ba None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:22:14.140582 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 741/1479] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:22:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:22:14.145986 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1521c8-b646-437e-842a-799073e70ed6 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:22:14.146419 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1521c8-b646-437e-842a-799073e70ed6 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:22:14.146730 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1521c8-b646-437e-842a-799073e70ed6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:22:14.213189 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-df1521c8-b646-437e-842a-799073e70ed6 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:22:14.214416 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-df1521c8-b646-437e-842a-799073e70ed6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:22:14.427472 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 739/1480] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:22:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:22:14.739992 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92da538c-b307-4e78-856b-b03ec1f0e1ba None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:22:14.739992 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-92da538c-b307-4e78-856b-b03ec1f0e1ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:22:14.739992 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:22:14.872354 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92da538c-b307-4e78-856b-b03ec1f0e1ba None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:22:14.952830 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:22:14.986610 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1da13d-6a29-4767-9e43-819c61019de8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f055f7dc51d43bda9b315ca67f11d58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:22:14.987160 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1da13d-6a29-4767-9e43-819c61019de8 service nova] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:22:14.987441 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1da13d-6a29-4767-9e43-819c61019de8 service nova] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:22:14.987757 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1da13d-6a29-4767-9e43-819c61019de8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:22:14.989369 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-ee1da13d-6a29-4767-9e43-819c61019de8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:22:15.077176 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1da13d-6a29-4767-9e43-819c61019de8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:22:15.078771 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1da13d-6a29-4767-9e43-819c61019de8 service nova] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:22:15.096514 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 742/1481] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:22:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:22:17.190813 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d0ae40-c9f6-4163-9805-dadf5ae02a2f None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:22:17.191379 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d0ae40-c9f6-4163-9805-dadf5ae02a2f None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:22:17.191379 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d0ae40-c9f6-4163-9805-dadf5ae02a2f None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:22:17.192648 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 740/1482] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:22:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:22:17.197835 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3591c508-ad6c-4e2c-a31a-3a3042128c73 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:22:17.198244 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3591c508-ad6c-4e2c-a31a-3a3042128c73 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:22:17.198378 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3591c508-ad6c-4e2c-a31a-3a3042128c73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:22:17.263500 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-3591c508-ad6c-4e2c-a31a-3a3042128c73 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:22:17.264720 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-3591c508-ad6c-4e2c-a31a-3a3042128c73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:22:17.320448 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:22:17.372945 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 743/1483] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:22:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:22:17.660007 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d0ae40-c9f6-4163-9805-dadf5ae02a2f None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:22:17.661587 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-02d0ae40-c9f6-4163-9805-dadf5ae02a2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:22:17.685297 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:22:17.764160 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d0ae40-c9f6-4163-9805-dadf5ae02a2f None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:22:17.836731 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a530465-c4ff-4cde-8954-5803f7e7e2e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e5260e798404e8bb12b25ca8ea97052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:22:17.837367 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a530465-c4ff-4cde-8954-5803f7e7e2e6 service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:22:17.837901 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a530465-c4ff-4cde-8954-5803f7e7e2e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:22:17.838172 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a530465-c4ff-4cde-8954-5803f7e7e2e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:22:17.839856 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-1a530465-c4ff-4cde-8954-5803f7e7e2e6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:22:17.911811 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:22:17.946919 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a530465-c4ff-4cde-8954-5803f7e7e2e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:22:17.948403 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a530465-c4ff-4cde-8954-5803f7e7e2e6 service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:22:17.966226 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 741/1484] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:22:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:22:19.939011 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae44b0b-6526-4446-a0a6-4d8b115de9ab None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:22:19.939613 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae44b0b-6526-4446-a0a6-4d8b115de9ab None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:22:19.939740 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae44b0b-6526-4446-a0a6-4d8b115de9ab None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:22:19.941016 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 744/1485] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:22:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:22:19.946389 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d9a201-de54-4305-83ad-399a92b17347 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:22:19.946816 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d9a201-de54-4305-83ad-399a92b17347 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:22:19.946951 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d9a201-de54-4305-83ad-399a92b17347 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:22:20.015513 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-30d9a201-de54-4305-83ad-399a92b17347 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:22:20.016854 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-30d9a201-de54-4305-83ad-399a92b17347 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:22:20.111755 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 742/1486] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:22:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:22:20.395538 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae44b0b-6526-4446-a0a6-4d8b115de9ab None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:22:20.397095 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-fae44b0b-6526-4446-a0a6-4d8b115de9ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:22:20.616832 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae44b0b-6526-4446-a0a6-4d8b115de9ab None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:22:20.686778 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:22:20.688672 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9f29a5-587d-4c11-997b-c1a5753f784d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e5260e798404e8bb12b25ca8ea97052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:22:20.689268 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9f29a5-587d-4c11-997b-c1a5753f784d service neutron] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:22:20.689543 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9f29a5-587d-4c11-997b-c1a5753f784d service neutron] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:22:20.689806 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9f29a5-587d-4c11-997b-c1a5753f784d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:22:20.691380 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-cb9f29a5-587d-4c11-997b-c1a5753f784d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:22:20.798372 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9f29a5-587d-4c11-997b-c1a5753f784d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:22:20.799814 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9f29a5-587d-4c11-997b-c1a5753f784d service neutron] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:22:20.817336 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 745/1487] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:22:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:22:22.947953 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c312ac-e05a-4e04-8b21-5c8f4a8912f4 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:22:22.948529 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c312ac-e05a-4e04-8b21-5c8f4a8912f4 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:22:22.948529 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c312ac-e05a-4e04-8b21-5c8f4a8912f4 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:22:22.949846 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 743/1488] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:22:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 09:22:22.955158 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e62b0d-b843-4e8d-9981-c70405b2015e None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:22:22.955610 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e62b0d-b843-4e8d-9981-c70405b2015e None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:22:22.955747 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e62b0d-b843-4e8d-9981-c70405b2015e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:22:23.020687 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-d1e62b0d-b843-4e8d-9981-c70405b2015e None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:22:23.021899 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-d1e62b0d-b843-4e8d-9981-c70405b2015e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:22:23.080480 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:22:23.125225 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 746/1489] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:22:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:22:23.419957 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c312ac-e05a-4e04-8b21-5c8f4a8912f4 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:22:23.421562 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-11c312ac-e05a-4e04-8b21-5c8f4a8912f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:22:23.457215 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:22:23.523712 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c312ac-e05a-4e04-8b21-5c8f4a8912f4 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:22:23.595343 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6c6f9f-2a73-49d7-b59d-da0334c56370 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e5260e798404e8bb12b25ca8ea97052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:22:23.595970 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6c6f9f-2a73-49d7-b59d-da0334c56370 service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:22:23.596252 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6c6f9f-2a73-49d7-b59d-da0334c56370 service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:22:23.596519 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6c6f9f-2a73-49d7-b59d-da0334c56370 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:22:23.598118 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-fa6c6f9f-2a73-49d7-b59d-da0334c56370 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:22:23.675776 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:22:23.704081 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6c6f9f-2a73-49d7-b59d-da0334c56370 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:22:23.705510 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6c6f9f-2a73-49d7-b59d-da0334c56370 service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:22:23.723841 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 744/1490] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:22:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:22:25.990097 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cf41b3-b75b-49f9-99b0-ea6c7c4a6d71 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:22:25.990697 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cf41b3-b75b-49f9-99b0-ea6c7c4a6d71 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:22:25.990697 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cf41b3-b75b-49f9-99b0-ea6c7c4a6d71 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:22:25.992059 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 747/1491] 10.222.0.16 () {54 vars in 927 bytes} [Wed Aug 24 09:22:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:22:25.997979 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23127fad-e88e-4610-94e8-c11997c566fe None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:22:25.998386 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23127fad-e88e-4610-94e8-c11997c566fe None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:22:25.998514 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23127fad-e88e-4610-94e8-c11997c566fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:22:26.063843 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-23127fad-e88e-4610-94e8-c11997c566fe None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:22:26.065031 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-23127fad-e88e-4610-94e8-c11997c566fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:22:26.159144 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 745/1492] 10.222.0.16 () {58 vars in 1024 bytes} [Wed Aug 24 09:22:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:22:26.451527 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cf41b3-b75b-49f9-99b0-ea6c7c4a6d71 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:22:26.453085 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-c1cf41b3-b75b-49f9-99b0-ea6c7c4a6d71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:22:26.559738 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cf41b3-b75b-49f9-99b0-ea6c7c4a6d71 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:22:26.631964 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f06a5a-c7cc-43f2-92e4-04d7562e4605 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e5260e798404e8bb12b25ca8ea97052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:22:26.632560 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f06a5a-c7cc-43f2-92e4-04d7562e4605 service neutron] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:22:26.632834 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f06a5a-c7cc-43f2-92e4-04d7562e4605 service neutron] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:22:26.633098 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f06a5a-c7cc-43f2-92e4-04d7562e4605 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:22:26.635449 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-f6f06a5a-c7cc-43f2-92e4-04d7562e4605 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:22:26.644327 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:22:26.744145 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f06a5a-c7cc-43f2-92e4-04d7562e4605 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:22:26.745579 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f06a5a-c7cc-43f2-92e4-04d7562e4605 service neutron] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:22:26.764141 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 748/1493] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:22:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:22:36.157372 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=82 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:22:36.761882 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=50 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:24:15.649609 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfd1d08-79d6-4834-b2a9-df164cc4ffd1 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:24:15.650209 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfd1d08-79d6-4834-b2a9-df164cc4ffd1 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:24:15.650209 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfd1d08-79d6-4834-b2a9-df164cc4ffd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:24:15.732314 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-9cfd1d08-79d6-4834-b2a9-df164cc4ffd1 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:24:15.733510 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-9cfd1d08-79d6-4834-b2a9-df164cc4ffd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:24:15.836713 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 746/1494] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 09:24:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:24:15.846118 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8c1d98-0805-42bc-b99c-e9dc0e9f3eb0 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:24:15.846654 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8c1d98-0805-42bc-b99c-e9dc0e9f3eb0 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:24:15.846654 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8c1d98-0805-42bc-b99c-e9dc0e9f3eb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:24:15.866103 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-aa8c1d98-0805-42bc-b99c-e9dc0e9f3eb0 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:24:15.867301 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-aa8c1d98-0805-42bc-b99c-e9dc0e9f3eb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:24:15.876429 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 749/1495] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 09:24:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:24:15.880663 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfd1d08-79d6-4834-b2a9-df164cc4ffd1 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:24:15.892377 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfd1d08-79d6-4834-b2a9-df164cc4ffd1 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:24:15.897688 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80532bc-391b-42be-9af1-26d71a1bc128 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:24:15.898332 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80532bc-391b-42be-9af1-26d71a1bc128 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:24:15.898615 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80532bc-391b-42be-9af1-26d71a1bc128 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:24:15.898883 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80532bc-391b-42be-9af1-26d71a1bc128 admin admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:24:15.899519 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80532bc-391b-42be-9af1-26d71a1bc128 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:24:15.899898 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80532bc-391b-42be-9af1-26d71a1bc128 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:24:15.901008 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: WARNING py.warnings [None req-e80532bc-391b-42be-9af1-26d71a1bc128 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 24 09:24:15.901008 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: warnings.warn(msg) Aug 24 09:24:15.901008 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: Aug 24 09:24:15.902060 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80532bc-391b-42be-9af1-26d71a1bc128 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:24:15.913618 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 747/1496] 10.222.0.16 () {58 vars in 1153 bytes} [Wed Aug 24 09:24:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:24:15.921965 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8c1d98-0805-42bc-b99c-e9dc0e9f3eb0 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:24:15.931204 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8c1d98-0805-42bc-b99c-e9dc0e9f3eb0 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:24:15.936350 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c8775b-40c5-4503-abb6-d08543aac002 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:24:15.936945 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c8775b-40c5-4503-abb6-d08543aac002 admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:24:15.937223 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c8775b-40c5-4503-abb6-d08543aac002 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:24:15.937494 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c8775b-40c5-4503-abb6-d08543aac002 admin admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:24:15.938140 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c8775b-40c5-4503-abb6-d08543aac002 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:24:15.939866 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: WARNING py.warnings [None req-09c8775b-40c5-4503-abb6-d08543aac002 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 09:24:15.939866 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: warnings.warn(msg) Aug 24 09:24:15.939866 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: Aug 24 09:24:15.941061 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c8775b-40c5-4503-abb6-d08543aac002 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:24:16.034437 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 750/1497] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 24 09:24:15 2022] POST /identity/v3/projects => generated 377 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:24:16.038616 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80532bc-391b-42be-9af1-26d71a1bc128 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:24:16.048545 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80532bc-391b-42be-9af1-26d71a1bc128 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:24:16.097281 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:24:16.103868 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f3e46c-d5b8-4fc4-8d33-05d3ca8cf1f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:24:16.104469 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f3e46c-d5b8-4fc4-8d33-05d3ca8cf1f7 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:24:16.104752 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f3e46c-d5b8-4fc4-8d33-05d3ca8cf1f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:24:16.105027 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f3e46c-d5b8-4fc4-8d33-05d3ca8cf1f7 admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:24:16.105706 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f3e46c-d5b8-4fc4-8d33-05d3ca8cf1f7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:24:16.106649 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: WARNING py.warnings [None req-b3f3e46c-d5b8-4fc4-8d33-05d3ca8cf1f7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 09:24:16.106649 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: warnings.warn(msg) Aug 24 09:24:16.106649 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: Aug 24 09:24:16.108424 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f3e46c-d5b8-4fc4-8d33-05d3ca8cf1f7 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:24:16.176309 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 748/1498] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 24 09:24:16 2022] POST /identity/v3/users => generated 289 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:24:16.180433 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c8775b-40c5-4503-abb6-d08543aac002 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:24:16.191314 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c8775b-40c5-4503-abb6-d08543aac002 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:24:16.195869 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25cebff-8882-4033-9a62-d521540dd4f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:24:16.196465 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25cebff-8882-4033-9a62-d521540dd4f3 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:24:16.196744 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25cebff-8882-4033-9a62-d521540dd4f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:24:16.197016 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25cebff-8882-4033-9a62-d521540dd4f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:24:16.197654 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25cebff-8882-4033-9a62-d521540dd4f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:24:16.198771 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: WARNING py.warnings [None req-b25cebff-8882-4033-9a62-d521540dd4f3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 24 09:24:16.198771 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: warnings.warn(msg) Aug 24 09:24:16.198771 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: Aug 24 09:24:16.199703 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25cebff-8882-4033-9a62-d521540dd4f3 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:24:16.213433 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 751/1499] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 24 09:24:16 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:24:16.217881 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f3e46c-d5b8-4fc4-8d33-05d3ca8cf1f7 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:24:16.228395 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f3e46c-d5b8-4fc4-8d33-05d3ca8cf1f7 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:24:16.233372 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d65814-dbeb-4e40-ba1d-fbca605817a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:24:16.233997 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d65814-dbeb-4e40-ba1d-fbca605817a8 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:24:16.234343 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d65814-dbeb-4e40-ba1d-fbca605817a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:24:16.234682 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d65814-dbeb-4e40-ba1d-fbca605817a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:24:16.235400 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d65814-dbeb-4e40-ba1d-fbca605817a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:24:16.236450 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: WARNING py.warnings [None req-74d65814-dbeb-4e40-ba1d-fbca605817a8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 24 09:24:16.236450 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: warnings.warn(msg) Aug 24 09:24:16.236450 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: Aug 24 09:24:16.237305 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d65814-dbeb-4e40-ba1d-fbca605817a8 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:24:16.251195 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 749/1500] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 24 09:24:16 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:24:16.255363 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25cebff-8882-4033-9a62-d521540dd4f3 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:24:16.265727 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25cebff-8882-4033-9a62-d521540dd4f3 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:24:16.270283 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12aeb7a-2689-4bea-a91c-d17a987131e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:24:16.270930 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12aeb7a-2689-4bea-a91c-d17a987131e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:24:16.271210 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12aeb7a-2689-4bea-a91c-d17a987131e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:24:16.271486 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12aeb7a-2689-4bea-a91c-d17a987131e4 admin admin] PATH_INFO: `/v3/projects/0361deb9586a4115b62c39a5e87865a6/users/41f96adeb09e49b0b300ba68fe54a5ed/roles/136cdf78e00a426997dd4e37faff489b` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:24:16.286349 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12aeb7a-2689-4bea-a91c-d17a987131e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0361deb9586a4115b62c39a5e87865a6, user_id=41f96adeb09e49b0b300ba68fe54a5ed, role_id=136cdf78e00a426997dd4e37faff489b)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:24:16.287365 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: WARNING py.warnings [None req-a12aeb7a-2689-4bea-a91c-d17a987131e4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 09:24:16.287365 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: warnings.warn(msg) Aug 24 09:24:16.287365 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: Aug 24 09:24:16.289346 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12aeb7a-2689-4bea-a91c-d17a987131e4 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:24:16.364751 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 752/1501] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 24 09:24:16 2022] PUT /identity/v3/projects/0361deb9586a4115b62c39a5e87865a6/users/41f96adeb09e49b0b300ba68fe54a5ed/roles/136cdf78e00a426997dd4e37faff489b => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:24:16.376920 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff19cc4d-3f0d-4ada-b901-153a7eb933b8 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:24:16.377198 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff19cc4d-3f0d-4ada-b901-153a7eb933b8 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:24:16.377491 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff19cc4d-3f0d-4ada-b901-153a7eb933b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:24:16.408892 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-ff19cc4d-3f0d-4ada-b901-153a7eb933b8 None None] MFA Rules not processed for user `41f96adeb09e49b0b300ba68fe54a5ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:24:16.410149 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-ff19cc4d-3f0d-4ada-b901-153a7eb933b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:24:16.473472 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 750/1502] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 09:24:16 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:24:16.487794 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817b78dd-4a9a-4a57-a284-885c92ca76d6 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:24:16.488085 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817b78dd-4a9a-4a57-a284-885c92ca76d6 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:24:16.488362 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817b78dd-4a9a-4a57-a284-885c92ca76d6 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:24:16.489691 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 753/1503] 10.222.0.16 () {54 vars in 964 bytes} [Wed Aug 24 09:24:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:24:16.494369 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b0c1f4-6fe4-4653-aeec-ec04c33f58d4 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:24:16.494655 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b0c1f4-6fe4-4653-aeec-ec04c33f58d4 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:24:16.494930 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b0c1f4-6fe4-4653-aeec-ec04c33f58d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:24:16.552659 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-35b0c1f4-6fe4-4653-aeec-ec04c33f58d4 None None] MFA Rules not processed for user `05c32b83577c46488906fe409a790b24`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:24:16.553867 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-35b0c1f4-6fe4-4653-aeec-ec04c33f58d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:24:16.631981 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 751/1504] 10.222.0.16 () {58 vars in 1061 bytes} [Wed Aug 24 09:24:16 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:24:16.639725 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817b78dd-4a9a-4a57-a284-885c92ca76d6 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:24:16.650303 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817b78dd-4a9a-4a57-a284-885c92ca76d6 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:24:16.654540 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15077b3-6980-4698-9536-c26e37126dbb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c32b83577c46488906fe409a790b24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:24:16.655200 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15077b3-6980-4698-9536-c26e37126dbb service cinder] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:24:16.655473 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15077b3-6980-4698-9536-c26e37126dbb service cinder] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:24:16.655848 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15077b3-6980-4698-9536-c26e37126dbb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:24:16.664260 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15077b3-6980-4698-9536-c26e37126dbb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:24:16.665716 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15077b3-6980-4698-9536-c26e37126dbb service cinder] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:24:16.675080 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 754/1505] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:24:16 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:24:16.728754 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa908f74-5446-492e-b94c-541f50a54837 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:24:16.728754 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa908f74-5446-492e-b94c-541f50a54837 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:24:16.728754 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa908f74-5446-492e-b94c-541f50a54837 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:24:16.728754 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 752/1506] 10.222.0.16 () {52 vars in 830 bytes} [Wed Aug 24 09:24:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:24:16.730539 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15077b3-6980-4698-9536-c26e37126dbb service cinder] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:24:16.795483 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15077b3-6980-4698-9536-c26e37126dbb service cinder] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:24:16.800143 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d1242b-3252-4373-807d-8214998cb4af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:24:16.800780 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d1242b-3252-4373-807d-8214998cb4af admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:24:16.801060 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d1242b-3252-4373-807d-8214998cb4af admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:24:16.801335 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d1242b-3252-4373-807d-8214998cb4af admin admin] PATH_INFO: `/v3/users/41f96adeb09e49b0b300ba68fe54a5ed` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:24:16.802476 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d1242b-3252-4373-807d-8214998cb4af admin admin] RBAC: Authorizing `identity:delete_user(user_id=41f96adeb09e49b0b300ba68fe54a5ed)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:24:16.803484 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: WARNING py.warnings [None req-44d1242b-3252-4373-807d-8214998cb4af admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 09:24:16.803484 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: warnings.warn(msg) Aug 24 09:24:16.803484 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: Aug 24 09:24:16.804741 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d1242b-3252-4373-807d-8214998cb4af admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:24:17.019727 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-44d1242b-3252-4373-807d-8214998cb4af admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '41f96adeb09e49b0b300ba68fe54a5ed'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:24:17.020779 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-44d1242b-3252-4373-807d-8214998cb4af admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '41f96adeb09e49b0b300ba68fe54a5ed'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:24:17.043379 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-44d1242b-3252-4373-807d-8214998cb4af admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '41f96adeb09e49b0b300ba68fe54a5ed'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:24:17.057691 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-44d1242b-3252-4373-807d-8214998cb4af admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '41f96adeb09e49b0b300ba68fe54a5ed'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:24:17.062621 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:24:17.135855 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-44d1242b-3252-4373-807d-8214998cb4af admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '41f96adeb09e49b0b300ba68fe54a5ed'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:24:17.142730 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 755/1507] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 24 09:24:16 2022] DELETE /identity/v3/users/41f96adeb09e49b0b300ba68fe54a5ed => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:24:17.147128 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3bd514-eba6-4d57-9594-bb98a24933e2 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:24:17.147455 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3bd514-eba6-4d57-9594-bb98a24933e2 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:24:17.147745 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3bd514-eba6-4d57-9594-bb98a24933e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:24:17.167026 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-8c3bd514-eba6-4d57-9594-bb98a24933e2 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:24:17.168267 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-8c3bd514-eba6-4d57-9594-bb98a24933e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:24:17.177317 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:24:17.227651 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 753/1508] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 09:24:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:24:17.237885 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d1242b-3252-4373-807d-8214998cb4af admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:24:17.239279 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-44d1242b-3252-4373-807d-8214998cb4af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:24:17.316563 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d1242b-3252-4373-807d-8214998cb4af admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:24:17.320224 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28322f8c-e43a-44ad-af5d-dd1df2c9b971 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e5260e798404e8bb12b25ca8ea97052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:24:17.320844 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28322f8c-e43a-44ad-af5d-dd1df2c9b971 service neutron] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:24:17.321127 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28322f8c-e43a-44ad-af5d-dd1df2c9b971 service neutron] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:24:17.321398 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28322f8c-e43a-44ad-af5d-dd1df2c9b971 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:24:17.331046 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28322f8c-e43a-44ad-af5d-dd1df2c9b971 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:24:17.332518 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28322f8c-e43a-44ad-af5d-dd1df2c9b971 service neutron] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:24:17.342194 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 756/1509] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:24:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:24:17.818432 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3bd514-eba6-4d57-9594-bb98a24933e2 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:24:17.819931 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-8c3bd514-eba6-4d57-9594-bb98a24933e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:24:17.832116 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3bd514-eba6-4d57-9594-bb98a24933e2 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:24:17.838033 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bde6a33-86b2-4107-a182-41a56cc0c887 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:24:17.838849 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bde6a33-86b2-4107-a182-41a56cc0c887 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:24:17.839306 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bde6a33-86b2-4107-a182-41a56cc0c887 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:24:17.839603 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bde6a33-86b2-4107-a182-41a56cc0c887 admin admin] PATH_INFO: `/v3/projects/0361deb9586a4115b62c39a5e87865a6` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:24:17.840746 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bde6a33-86b2-4107-a182-41a56cc0c887 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0361deb9586a4115b62c39a5e87865a6)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:24:17.842394 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: WARNING py.warnings [None req-1bde6a33-86b2-4107-a182-41a56cc0c887 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 09:24:17.842394 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: warnings.warn(msg) Aug 24 09:24:17.842394 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: Aug 24 09:24:17.843927 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bde6a33-86b2-4107-a182-41a56cc0c887 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:24:17.953322 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-1bde6a33-86b2-4107-a182-41a56cc0c887 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0361deb9586a4115b62c39a5e87865a6', 'request_id': 'req-1bde6a33-86b2-4107-a182-41a56cc0c887'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:24:17.966384 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-1bde6a33-86b2-4107-a182-41a56cc0c887 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0361deb9586a4115b62c39a5e87865a6', 'request_id': 'req-1bde6a33-86b2-4107-a182-41a56cc0c887'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:24:17.972905 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-1bde6a33-86b2-4107-a182-41a56cc0c887 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0361deb9586a4115b62c39a5e87865a6', 'request_id': 'req-1bde6a33-86b2-4107-a182-41a56cc0c887'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:24:18.038467 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-1bde6a33-86b2-4107-a182-41a56cc0c887 admin admin] The token cache is being invalidate because project 0361deb9586a4115b62c39a5e87865a6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81602) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:24:18.038762 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-1bde6a33-86b2-4107-a182-41a56cc0c887 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:24:18.039419 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-1bde6a33-86b2-4107-a182-41a56cc0c887 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:24:18.041037 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 754/1510] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 24 09:24:17 2022] DELETE /identity/v3/projects/0361deb9586a4115b62c39a5e87865a6 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:24:27.338011 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:DELETE=1,keystone:SELECT=37 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:24:28.035131 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=47,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:25:52.529755 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98de7564-55f3-4ddd-b70f-c1e015c295a1 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:25:52.530365 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98de7564-55f3-4ddd-b70f-c1e015c295a1 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:25:52.530365 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98de7564-55f3-4ddd-b70f-c1e015c295a1 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:25:52.531689 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 757/1511] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 09:25:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:25:52.536471 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb150d3-e7aa-4bfc-abed-8308cdb8c0f8 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:25:52.536915 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb150d3-e7aa-4bfc-abed-8308cdb8c0f8 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:25:52.537019 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb150d3-e7aa-4bfc-abed-8308cdb8c0f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:25:52.558535 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-abb150d3-e7aa-4bfc-abed-8308cdb8c0f8 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:25:52.558535 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-abb150d3-e7aa-4bfc-abed-8308cdb8c0f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:25:52.615308 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 755/1512] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 09:25:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:25:52.768831 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98de7564-55f3-4ddd-b70f-c1e015c295a1 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:25:52.770639 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-98de7564-55f3-4ddd-b70f-c1e015c295a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:25:52.870052 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98de7564-55f3-4ddd-b70f-c1e015c295a1 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:25:52.875387 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a6acf7-318e-458e-8636-6bededeadf98 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f055f7dc51d43bda9b315ca67f11d58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:25:52.876021 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a6acf7-318e-458e-8636-6bededeadf98 service nova] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:25:52.876298 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a6acf7-318e-458e-8636-6bededeadf98 service nova] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:25:52.876568 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a6acf7-318e-458e-8636-6bededeadf98 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:25:52.884440 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a6acf7-318e-458e-8636-6bededeadf98 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:25:52.886003 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a6acf7-318e-458e-8636-6bededeadf98 service nova] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:25:52.895119 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 758/1513] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:25:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:25:53.728853 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5fa038-a7c6-4207-97db-86d0454a35ed None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:25:53.729452 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5fa038-a7c6-4207-97db-86d0454a35ed None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:25:53.729452 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5fa038-a7c6-4207-97db-86d0454a35ed None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:25:53.730520 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 756/1514] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 09:25:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:25:53.735335 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0999c2c5-a011-448f-b001-014ad627dcd6 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:25:53.735763 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0999c2c5-a011-448f-b001-014ad627dcd6 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:25:53.735931 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0999c2c5-a011-448f-b001-014ad627dcd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:25:53.755458 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-0999c2c5-a011-448f-b001-014ad627dcd6 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:25:53.756688 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-0999c2c5-a011-448f-b001-014ad627dcd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:25:53.764421 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 759/1515] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 09:25:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:25:53.915353 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5fa038-a7c6-4207-97db-86d0454a35ed None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:25:53.930119 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5fa038-a7c6-4207-97db-86d0454a35ed None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:25:53.935548 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28639925-6c87-44c0-85a0-b67ade7a4992 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f055f7dc51d43bda9b315ca67f11d58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:25:53.936172 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28639925-6c87-44c0-85a0-b67ade7a4992 service nova] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:25:53.936453 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28639925-6c87-44c0-85a0-b67ade7a4992 service nova] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:25:53.936728 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28639925-6c87-44c0-85a0-b67ade7a4992 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:25:53.945363 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28639925-6c87-44c0-85a0-b67ade7a4992 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:25:53.946946 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28639925-6c87-44c0-85a0-b67ade7a4992 service nova] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:25:53.956464 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 757/1516] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:25:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:25:54.891484 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3322799-42cf-4ad0-bd66-9b5e7aa3f966 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:25:54.892046 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3322799-42cf-4ad0-bd66-9b5e7aa3f966 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:25:54.892046 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3322799-42cf-4ad0-bd66-9b5e7aa3f966 None None] PATH_INFO: `/v3` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:25:54.893338 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 760/1517] 10.222.0.17 () {54 vars in 925 bytes} [Wed Aug 24 09:25:54 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:25:54.899703 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b3844d-f777-4f55-a782-d5342e009007 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:25:54.900106 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b3844d-f777-4f55-a782-d5342e009007 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:25:54.900257 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b3844d-f777-4f55-a782-d5342e009007 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:25:54.949160 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-06b3844d-f777-4f55-a782-d5342e009007 None None] MFA Rules not processed for user `80cada22039d4f11935412ee578acd05`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:25:54.950388 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-06b3844d-f777-4f55-a782-d5342e009007 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:25:55.019673 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 758/1518] 10.222.0.17 () {58 vars in 1015 bytes} [Wed Aug 24 09:25:54 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:26:03.750289 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=49 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:26:04.674068 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755fee84-0032-461d-915c-e010ef8cde88 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:26:04.674355 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755fee84-0032-461d-915c-e010ef8cde88 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:26:04.674629 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755fee84-0032-461d-915c-e010ef8cde88 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:26:04.676032 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 761/1519] 10.222.0.16 () {54 vars in 961 bytes} [Wed Aug 24 09:26:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:26:04.681467 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83664f8f-fe89-4cac-bcdd-156299103666 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:26:04.681903 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83664f8f-fe89-4cac-bcdd-156299103666 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:26:04.682152 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83664f8f-fe89-4cac-bcdd-156299103666 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:26:04.701614 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-83664f8f-fe89-4cac-bcdd-156299103666 None None] MFA Rules not processed for user `80cada22039d4f11935412ee578acd05`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:26:04.702820 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-83664f8f-fe89-4cac-bcdd-156299103666 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:26:04.711282 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 759/1520] 10.222.0.16 () {58 vars in 1058 bytes} [Wed Aug 24 09:26:04 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:26:04.719674 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755fee84-0032-461d-915c-e010ef8cde88 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:26:04.733265 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755fee84-0032-461d-915c-e010ef8cde88 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:26:04.738260 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98483320-5892-4ada-9b9c-2d34e2ed506d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80cada22039d4f11935412ee578acd05', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:26:04.738860 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98483320-5892-4ada-9b9c-2d34e2ed506d service placement] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:26:04.739140 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98483320-5892-4ada-9b9c-2d34e2ed506d service placement] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:26:04.739412 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98483320-5892-4ada-9b9c-2d34e2ed506d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:26:04.748652 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98483320-5892-4ada-9b9c-2d34e2ed506d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:26:04.750241 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98483320-5892-4ada-9b9c-2d34e2ed506d service placement] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:26:04.760355 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 762/1521] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:26:04 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:26:05.189938 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a1bac2-ac3d-492b-985b-89c38466363d None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:26:05.190289 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a1bac2-ac3d-492b-985b-89c38466363d None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:26:05.190565 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a1bac2-ac3d-492b-985b-89c38466363d None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:26:05.191934 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 760/1522] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 09:26:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:26:05.196877 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d917ee1c-27e3-4978-a930-3515eb0128c9 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:26:05.197164 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d917ee1c-27e3-4978-a930-3515eb0128c9 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:26:05.197442 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d917ee1c-27e3-4978-a930-3515eb0128c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:26:05.217144 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-d917ee1c-27e3-4978-a930-3515eb0128c9 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:26:05.221310 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-d917ee1c-27e3-4978-a930-3515eb0128c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:26:05.226771 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 763/1523] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 09:26:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:26:05.389332 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a1bac2-ac3d-492b-985b-89c38466363d None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:26:05.404102 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a1bac2-ac3d-492b-985b-89c38466363d None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:26:05.409664 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aed93f9-eb94-4591-bca2-7626dd9740c9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f055f7dc51d43bda9b315ca67f11d58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:26:05.410337 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aed93f9-eb94-4591-bca2-7626dd9740c9 service nova] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:26:05.410639 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aed93f9-eb94-4591-bca2-7626dd9740c9 service nova] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:26:05.410914 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aed93f9-eb94-4591-bca2-7626dd9740c9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:26:05.419393 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aed93f9-eb94-4591-bca2-7626dd9740c9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:26:05.421272 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aed93f9-eb94-4591-bca2-7626dd9740c9 service nova] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:26:05.431014 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 761/1524] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:26:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:26:06.265680 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7d0a7e-8597-41e1-a364-95f6687dd2c3 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:26:06.266234 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7d0a7e-8597-41e1-a364-95f6687dd2c3 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:26:06.266234 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7d0a7e-8597-41e1-a364-95f6687dd2c3 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:26:06.267565 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 764/1525] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 09:26:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:26:06.272486 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7efe99-1491-4885-8103-5c3f34e34128 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:26:06.272893 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7efe99-1491-4885-8103-5c3f34e34128 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:26:06.273117 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7efe99-1491-4885-8103-5c3f34e34128 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:26:06.292372 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-1f7efe99-1491-4885-8103-5c3f34e34128 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:26:06.293566 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-1f7efe99-1491-4885-8103-5c3f34e34128 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:26:06.301281 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 762/1526] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 09:26:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:26:06.456697 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7d0a7e-8597-41e1-a364-95f6687dd2c3 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:26:06.471598 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7d0a7e-8597-41e1-a364-95f6687dd2c3 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:26:06.477516 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4caf99b1-d1c6-4866-87a4-e698494e36c3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f055f7dc51d43bda9b315ca67f11d58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:26:06.478105 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caf99b1-d1c6-4866-87a4-e698494e36c3 service nova] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:26:06.478381 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caf99b1-d1c6-4866-87a4-e698494e36c3 service nova] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:26:06.478648 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caf99b1-d1c6-4866-87a4-e698494e36c3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:26:06.486447 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4caf99b1-d1c6-4866-87a4-e698494e36c3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:26:06.488034 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4caf99b1-d1c6-4866-87a4-e698494e36c3 service nova] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:26:06.497802 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 765/1527] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:26:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:26:10.436133 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd319574-1edc-45b2-bd53-7ff48a3f6063 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:26:10.436696 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd319574-1edc-45b2-bd53-7ff48a3f6063 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:26:10.436696 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd319574-1edc-45b2-bd53-7ff48a3f6063 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:26:10.438286 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 763/1528] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 09:26:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:26:10.444184 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893aca7a-c352-4179-933c-e21f01a37f9f None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:26:10.444616 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893aca7a-c352-4179-933c-e21f01a37f9f None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:26:10.444726 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893aca7a-c352-4179-933c-e21f01a37f9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:26:10.464124 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-893aca7a-c352-4179-933c-e21f01a37f9f None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:26:10.465316 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-893aca7a-c352-4179-933c-e21f01a37f9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:26:10.473124 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 766/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 09:26:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 09:26:10.755900 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd319574-1edc-45b2-bd53-7ff48a3f6063 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:26:10.757648 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-bd319574-1edc-45b2-bd53-7ff48a3f6063 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:26:10.787287 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:26:10.817204 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd319574-1edc-45b2-bd53-7ff48a3f6063 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:26:10.820825 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4797a3b3-3ef1-4fc9-9eb1-c6a6c4e40392 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e5260e798404e8bb12b25ca8ea97052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:26:10.821618 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4797a3b3-3ef1-4fc9-9eb1-c6a6c4e40392 service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:26:10.821899 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4797a3b3-3ef1-4fc9-9eb1-c6a6c4e40392 service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:26:10.822170 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4797a3b3-3ef1-4fc9-9eb1-c6a6c4e40392 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:26:10.831540 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4797a3b3-3ef1-4fc9-9eb1-c6a6c4e40392 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:26:10.832980 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4797a3b3-3ef1-4fc9-9eb1-c6a6c4e40392 service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:26:10.842528 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 764/1530] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:26:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:26:12.733688 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db02cb29-54e3-4cad-91fa-32580f5bed93 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:26:12.734238 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db02cb29-54e3-4cad-91fa-32580f5bed93 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:26:12.734238 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db02cb29-54e3-4cad-91fa-32580f5bed93 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:26:12.735611 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 767/1531] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 09:26:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:26:12.741208 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b076c6d2-6cd1-4519-a4c1-fb75bfaad084 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:26:12.741632 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b076c6d2-6cd1-4519-a4c1-fb75bfaad084 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:26:12.741750 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b076c6d2-6cd1-4519-a4c1-fb75bfaad084 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:26:12.761853 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-b076c6d2-6cd1-4519-a4c1-fb75bfaad084 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:26:12.763066 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-b076c6d2-6cd1-4519-a4c1-fb75bfaad084 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:26:12.771136 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 765/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 09:26:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:26:13.053606 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db02cb29-54e3-4cad-91fa-32580f5bed93 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:26:13.055293 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-db02cb29-54e3-4cad-91fa-32580f5bed93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:26:13.067896 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db02cb29-54e3-4cad-91fa-32580f5bed93 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:26:13.071523 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edad925d-d4fe-4cf1-8bc8-4e22756d59b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e5260e798404e8bb12b25ca8ea97052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:26:13.072150 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edad925d-d4fe-4cf1-8bc8-4e22756d59b9 service neutron] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:26:13.072427 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edad925d-d4fe-4cf1-8bc8-4e22756d59b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:26:13.072696 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edad925d-d4fe-4cf1-8bc8-4e22756d59b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:26:13.082039 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edad925d-d4fe-4cf1-8bc8-4e22756d59b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:26:13.083445 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edad925d-d4fe-4cf1-8bc8-4e22756d59b9 service neutron] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:26:13.093969 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 768/1533] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:26:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:26:16.623429 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b0dbda-ab2d-4ccb-80a2-547efa3f456f None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:26:16.623975 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b0dbda-ab2d-4ccb-80a2-547efa3f456f None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:26:16.624092 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b0dbda-ab2d-4ccb-80a2-547efa3f456f None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:26:16.625471 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 766/1534] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 09:26:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:26:16.631636 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11bf700-7424-4a6a-b883-d27797abb61c None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:26:16.638440 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11bf700-7424-4a6a-b883-d27797abb61c None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:26:16.638440 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11bf700-7424-4a6a-b883-d27797abb61c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:26:16.654294 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-f11bf700-7424-4a6a-b883-d27797abb61c None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:26:16.655511 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-f11bf700-7424-4a6a-b883-d27797abb61c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:26:16.663568 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 769/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 09:26:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:26:17.217158 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b0dbda-ab2d-4ccb-80a2-547efa3f456f None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:26:17.218807 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-60b0dbda-ab2d-4ccb-80a2-547efa3f456f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:26:17.275918 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b0dbda-ab2d-4ccb-80a2-547efa3f456f None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:26:17.279513 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c59b7f-725c-4989-8a07-0ee9d02d3e8d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c32b83577c46488906fe409a790b24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:26:17.280140 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c59b7f-725c-4989-8a07-0ee9d02d3e8d service cinder] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:26:17.280421 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c59b7f-725c-4989-8a07-0ee9d02d3e8d service cinder] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:26:17.280694 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c59b7f-725c-4989-8a07-0ee9d02d3e8d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:26:17.290209 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c59b7f-725c-4989-8a07-0ee9d02d3e8d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:26:17.291773 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c59b7f-725c-4989-8a07-0ee9d02d3e8d service cinder] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:26:17.301611 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 767/1536] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:26:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:26:19.225094 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6d21de-9d06-4f02-9859-4e075ae86379 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:26:19.225640 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6d21de-9d06-4f02-9859-4e075ae86379 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:26:19.225640 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6d21de-9d06-4f02-9859-4e075ae86379 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:26:19.226993 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 770/1537] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 09:26:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 09:26:19.232366 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314fad87-fa0d-4c01-b149-d41834206b97 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:26:19.232815 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314fad87-fa0d-4c01-b149-d41834206b97 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:26:19.232931 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314fad87-fa0d-4c01-b149-d41834206b97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:26:19.251855 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-314fad87-fa0d-4c01-b149-d41834206b97 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:26:19.253034 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-314fad87-fa0d-4c01-b149-d41834206b97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:26:19.260838 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 768/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 09:26:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:26:19.808913 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6d21de-9d06-4f02-9859-4e075ae86379 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:26:19.810761 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-dc6d21de-9d06-4f02-9859-4e075ae86379 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:26:19.824145 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6d21de-9d06-4f02-9859-4e075ae86379 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:26:19.827749 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073accf3-0fb6-4e6e-8138-79d8a0bf9017 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c32b83577c46488906fe409a790b24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:26:19.828390 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073accf3-0fb6-4e6e-8138-79d8a0bf9017 service cinder] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:26:19.828671 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073accf3-0fb6-4e6e-8138-79d8a0bf9017 service cinder] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:26:19.828945 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073accf3-0fb6-4e6e-8138-79d8a0bf9017 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:26:19.839004 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073accf3-0fb6-4e6e-8138-79d8a0bf9017 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:26:19.840500 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073accf3-0fb6-4e6e-8138-79d8a0bf9017 service cinder] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:26:19.849671 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 771/1539] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:26:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:26:29.246703 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=47 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:26:29.845605 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=33 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:27:46.429254 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dfca8c-7e63-4b8c-beca-cadf3b426f28 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:27:46.430326 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dfca8c-7e63-4b8c-beca-cadf3b426f28 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:27:46.430817 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dfca8c-7e63-4b8c-beca-cadf3b426f28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:27:46.447858 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8b911c-9445-434d-b7f9-cedeaff2c226 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:27:46.448896 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8b911c-9445-434d-b7f9-cedeaff2c226 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:27:46.449404 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8b911c-9445-434d-b7f9-cedeaff2c226 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:27:46.459244 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-17dfca8c-7e63-4b8c-beca-cadf3b426f28 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:27:46.460874 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-17dfca8c-7e63-4b8c-beca-cadf3b426f28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:27:46.478789 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 769/1540] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 09:27:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:27:46.486596 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-6a8b911c-9445-434d-b7f9-cedeaff2c226 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:27:46.487548 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-6a8b911c-9445-434d-b7f9-cedeaff2c226 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:27:46.505661 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 772/1541] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 09:27:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 09:27:46.507519 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f995e54-531e-4f3e-b1f7-96d65cc32973 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:27:46.508083 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f995e54-531e-4f3e-b1f7-96d65cc32973 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:27:46.508577 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f995e54-531e-4f3e-b1f7-96d65cc32973 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:27:46.514022 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d110cc18-6306-4ec8-b30f-1e301fa0a7ef None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:27:46.514685 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d110cc18-6306-4ec8-b30f-1e301fa0a7ef None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:27:46.515259 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d110cc18-6306-4ec8-b30f-1e301fa0a7ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:27:46.555271 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-d110cc18-6306-4ec8-b30f-1e301fa0a7ef None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:27:46.556567 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-3f995e54-531e-4f3e-b1f7-96d65cc32973 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:27:46.558344 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-3f995e54-531e-4f3e-b1f7-96d65cc32973 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:27:46.559455 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-d110cc18-6306-4ec8-b30f-1e301fa0a7ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:27:46.580353 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 770/1542] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 09:27:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:27:46.581646 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7020a6-5e13-41d3-85c2-153245fbc31a None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:27:46.581932 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7020a6-5e13-41d3-85c2-153245fbc31a None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:27:46.582209 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7020a6-5e13-41d3-85c2-153245fbc31a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:27:46.583494 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 773/1543] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 09:27:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:27:46.588249 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3455102d-e3e8-4f27-8731-a16254305adc None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:27:46.588786 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3455102d-e3e8-4f27-8731-a16254305adc None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:27:46.589275 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3455102d-e3e8-4f27-8731-a16254305adc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:27:46.604638 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-3a7020a6-5e13-41d3-85c2-153245fbc31a None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:27:46.606547 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-3a7020a6-5e13-41d3-85c2-153245fbc31a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:27:46.614672 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 771/1544] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 09:27:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:27:46.616694 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05467eb4-c5be-4fb4-985a-85a9ffa52de4 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:27:46.616979 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05467eb4-c5be-4fb4-985a-85a9ffa52de4 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:27:46.617256 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05467eb4-c5be-4fb4-985a-85a9ffa52de4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:27:46.623465 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-3455102d-e3e8-4f27-8731-a16254305adc None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:27:46.624590 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-3455102d-e3e8-4f27-8731-a16254305adc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:27:46.635347 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 774/1545] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 09:27:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:27:46.636852 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3455102d-e3e8-4f27-8731-a16254305adc None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:27:46.641939 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-05467eb4-c5be-4fb4-985a-85a9ffa52de4 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:27:46.643995 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-05467eb4-c5be-4fb4-985a-85a9ffa52de4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:27:46.650530 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3455102d-e3e8-4f27-8731-a16254305adc None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:27:46.652606 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 772/1546] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 09:27:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:27:46.655758 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05467eb4-c5be-4fb4-985a-85a9ffa52de4 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:27:46.657458 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c54f9c-a468-4ad2-9a6d-93f11faee5eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:27:46.658308 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c54f9c-a468-4ad2-9a6d-93f11faee5eb admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:27:46.658685 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c54f9c-a468-4ad2-9a6d-93f11faee5eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:27:46.659056 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c54f9c-a468-4ad2-9a6d-93f11faee5eb admin admin] PATH_INFO: `/v3/domains` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:27:46.659874 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c54f9c-a468-4ad2-9a6d-93f11faee5eb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:27:46.660277 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c54f9c-a468-4ad2-9a6d-93f11faee5eb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:27:46.661540 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: WARNING py.warnings [None req-13c54f9c-a468-4ad2-9a6d-93f11faee5eb admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 24 09:27:46.661540 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: warnings.warn(msg) Aug 24 09:27:46.661540 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: Aug 24 09:27:46.662483 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c54f9c-a468-4ad2-9a6d-93f11faee5eb admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:27:46.670107 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05467eb4-c5be-4fb4-985a-85a9ffa52de4 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:27:46.676062 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e045ab-0f5f-4033-984b-fd8f7cb6a58e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:27:46.683601 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 775/1547] 10.222.0.16 () {58 vars in 1153 bytes} [Wed Aug 24 09:27:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:27:46.685744 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e045ab-0f5f-4033-984b-fd8f7cb6a58e admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:27:46.685744 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e045ab-0f5f-4033-984b-fd8f7cb6a58e admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:27:46.685744 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e045ab-0f5f-4033-984b-fd8f7cb6a58e admin admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:27:46.686059 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ad5a57-abc9-4e0f-ac05-01c29b186e4d None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:27:46.686149 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e045ab-0f5f-4033-984b-fd8f7cb6a58e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:27:46.686239 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ad5a57-abc9-4e0f-ac05-01c29b186e4d None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:27:46.686332 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e045ab-0f5f-4033-984b-fd8f7cb6a58e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:27:46.686496 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ad5a57-abc9-4e0f-ac05-01c29b186e4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:27:46.686604 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e045ab-0f5f-4033-984b-fd8f7cb6a58e admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:27:46.693240 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 773/1548] 10.222.0.16 () {58 vars in 1153 bytes} [Wed Aug 24 09:27:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:27:46.694217 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf26fe4-3a3e-423f-8d67-60178202ca37 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:27:46.694503 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf26fe4-3a3e-423f-8d67-60178202ca37 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:27:46.694796 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf26fe4-3a3e-423f-8d67-60178202ca37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:27:46.704458 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-10ad5a57-abc9-4e0f-ac05-01c29b186e4d None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:27:46.705670 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-10ad5a57-abc9-4e0f-ac05-01c29b186e4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:27:46.714773 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 776/1549] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 09:27:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:27:46.716432 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd16b9e-b2fa-4b23-97e2-94a7d2e4c67f None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:27:46.716943 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd16b9e-b2fa-4b23-97e2-94a7d2e4c67f None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:27:46.717076 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-ccf26fe4-3a3e-423f-8d67-60178202ca37 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:27:46.717661 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd16b9e-b2fa-4b23-97e2-94a7d2e4c67f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:27:46.718241 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-ccf26fe4-3a3e-423f-8d67-60178202ca37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:27:46.726769 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 774/1550] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 09:27:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:27:46.728473 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c29a9c-7aa8-45cf-966a-6548ef37576c None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:27:46.728795 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c29a9c-7aa8-45cf-966a-6548ef37576c None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:27:46.729142 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c29a9c-7aa8-45cf-966a-6548ef37576c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:27:46.741215 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-ebd16b9e-b2fa-4b23-97e2-94a7d2e4c67f None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:27:46.742480 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-ebd16b9e-b2fa-4b23-97e2-94a7d2e4c67f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:27:46.748689 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-56c29a9c-7aa8-45cf-966a-6548ef37576c None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:27:46.749854 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-56c29a9c-7aa8-45cf-966a-6548ef37576c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:27:46.750816 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 777/1551] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 09:27:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:27:46.752092 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5cde8c-38c2-4072-8771-ac1a8bcfccd8 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:27:46.752378 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5cde8c-38c2-4072-8771-ac1a8bcfccd8 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:27:46.752647 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5cde8c-38c2-4072-8771-ac1a8bcfccd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:27:46.758403 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 775/1552] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 09:27:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:27:46.759800 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d24719-8e8a-49e8-bad6-e829ce1dc6cc None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:27:46.760086 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d24719-8e8a-49e8-bad6-e829ce1dc6cc None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:27:46.760362 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d24719-8e8a-49e8-bad6-e829ce1dc6cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:27:46.777101 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-fd5cde8c-38c2-4072-8771-ac1a8bcfccd8 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:27:46.778337 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-fd5cde8c-38c2-4072-8771-ac1a8bcfccd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:27:46.779722 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-f4d24719-8e8a-49e8-bad6-e829ce1dc6cc None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:27:46.780888 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-f4d24719-8e8a-49e8-bad6-e829ce1dc6cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:27:46.786044 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 778/1553] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 09:27:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:27:46.786784 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5cde8c-38c2-4072-8771-ac1a8bcfccd8 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:27:46.790562 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 776/1554] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 09:27:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:27:46.790562 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d24719-8e8a-49e8-bad6-e829ce1dc6cc None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:27:46.797108 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5cde8c-38c2-4072-8771-ac1a8bcfccd8 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:27:46.801211 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d24719-8e8a-49e8-bad6-e829ce1dc6cc None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:27:46.803080 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb51bea-634f-48b1-ac9d-62a93498b140 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:27:46.803908 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb51bea-634f-48b1-ac9d-62a93498b140 admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:27:46.804476 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb51bea-634f-48b1-ac9d-62a93498b140 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:27:46.804916 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb51bea-634f-48b1-ac9d-62a93498b140 admin admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:27:46.805760 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb51bea-634f-48b1-ac9d-62a93498b140 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:27:46.806441 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6987344c-47e4-4eed-92b1-570427041644 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:27:46.807048 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6987344c-47e4-4eed-92b1-570427041644 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:27:46.807325 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6987344c-47e4-4eed-92b1-570427041644 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:27:46.807622 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6987344c-47e4-4eed-92b1-570427041644 admin admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:27:46.807751 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb51bea-634f-48b1-ac9d-62a93498b140 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:27:46.808238 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6987344c-47e4-4eed-92b1-570427041644 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:27:46.809231 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: WARNING py.warnings [None req-6987344c-47e4-4eed-92b1-570427041644 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 09:27:46.809231 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: warnings.warn(msg) Aug 24 09:27:46.809231 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: Aug 24 09:27:46.811188 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6987344c-47e4-4eed-92b1-570427041644 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:27:46.882283 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 777/1555] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 24 09:27:46 2022] POST /identity/v3/projects => generated 381 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:27:46.883463 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 779/1556] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 24 09:27:46 2022] POST /identity/v3/projects => generated 373 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:27:46.884053 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f904330f-d469-413b-a7c3-bafd6c76d2fe None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:27:46.884519 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f904330f-d469-413b-a7c3-bafd6c76d2fe None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:27:46.884651 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb51bea-634f-48b1-ac9d-62a93498b140 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:27:46.885178 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f904330f-d469-413b-a7c3-bafd6c76d2fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:27:46.906526 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-f904330f-d469-413b-a7c3-bafd6c76d2fe None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:27:46.907758 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-f904330f-d469-413b-a7c3-bafd6c76d2fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:27:46.949617 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb51bea-634f-48b1-ac9d-62a93498b140 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:27:46.954242 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9feb566-dab3-46e4-8eeb-45f87cffdcfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:27:46.954851 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9feb566-dab3-46e4-8eeb-45f87cffdcfc admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:27:46.955127 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9feb566-dab3-46e4-8eeb-45f87cffdcfc admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:27:46.955399 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9feb566-dab3-46e4-8eeb-45f87cffdcfc admin admin] PATH_INFO: `/v3/domains` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:27:46.956073 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9feb566-dab3-46e4-8eeb-45f87cffdcfc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:27:46.956396 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9feb566-dab3-46e4-8eeb-45f87cffdcfc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:27:46.957972 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9feb566-dab3-46e4-8eeb-45f87cffdcfc admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:27:46.968764 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 778/1557] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 09:27:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:27:46.970023 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598263fa-d89f-4376-86ce-42680fe103fa None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:27:46.970310 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598263fa-d89f-4376-86ce-42680fe103fa None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:27:46.970583 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598263fa-d89f-4376-86ce-42680fe103fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:27:46.970828 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 780/1558] 10.222.0.16 () {58 vars in 1153 bytes} [Wed Aug 24 09:27:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:27:46.971685 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9feb566-dab3-46e4-8eeb-45f87cffdcfc admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:27:46.982185 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9feb566-dab3-46e4-8eeb-45f87cffdcfc admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:27:46.986845 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e68a53-5d93-472c-a3d7-155d514640e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:27:46.987450 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e68a53-5d93-472c-a3d7-155d514640e9 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:27:46.987770 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e68a53-5d93-472c-a3d7-155d514640e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:27:46.988045 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e68a53-5d93-472c-a3d7-155d514640e9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:27:46.988687 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e68a53-5d93-472c-a3d7-155d514640e9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:27:46.988997 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e68a53-5d93-472c-a3d7-155d514640e9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:27:46.990597 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e68a53-5d93-472c-a3d7-155d514640e9 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:27:46.991305 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-598263fa-d89f-4376-86ce-42680fe103fa None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:27:46.992572 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-598263fa-d89f-4376-86ce-42680fe103fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:27:47.000580 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 779/1559] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 09:27:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:27:47.001368 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598263fa-d89f-4376-86ce-42680fe103fa None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:27:47.009829 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 781/1560] 10.222.0.16 () {58 vars in 1153 bytes} [Wed Aug 24 09:27:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:27:47.009829 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3511d9-e42e-4b92-89be-59689c54f4f3 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:27:47.009829 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3511d9-e42e-4b92-89be-59689c54f4f3 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:27:47.009829 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3511d9-e42e-4b92-89be-59689c54f4f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:27:47.015550 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598263fa-d89f-4376-86ce-42680fe103fa None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:27:47.020484 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80724383-94ac-482b-ab75-d27d4b69d3a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:27:47.021115 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80724383-94ac-482b-ab75-d27d4b69d3a0 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:27:47.021380 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80724383-94ac-482b-ab75-d27d4b69d3a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:27:47.021655 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80724383-94ac-482b-ab75-d27d4b69d3a0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:27:47.022306 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80724383-94ac-482b-ab75-d27d4b69d3a0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:27:47.022610 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80724383-94ac-482b-ab75-d27d4b69d3a0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:27:47.023642 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-ca3511d9-e42e-4b92-89be-59689c54f4f3 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:27:47.024423 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80724383-94ac-482b-ab75-d27d4b69d3a0 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:27:47.024844 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-ca3511d9-e42e-4b92-89be-59689c54f4f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:27:47.033946 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 782/1561] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 09:27:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:27:47.035540 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3511d9-e42e-4b92-89be-59689c54f4f3 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:27:47.046812 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 780/1562] 10.222.0.16 () {58 vars in 1153 bytes} [Wed Aug 24 09:27:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:27:47.046812 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80724383-94ac-482b-ab75-d27d4b69d3a0 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:27:47.049845 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3511d9-e42e-4b92-89be-59689c54f4f3 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:27:47.053602 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80724383-94ac-482b-ab75-d27d4b69d3a0 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:27:47.054510 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9eb8a1-5ce4-48ed-873c-1e2f845f6b2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:27:47.055120 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9eb8a1-5ce4-48ed-873c-1e2f845f6b2b admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:27:47.055506 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9eb8a1-5ce4-48ed-873c-1e2f845f6b2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:27:47.059095 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9eb8a1-5ce4-48ed-873c-1e2f845f6b2b admin admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:27:47.059095 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9eb8a1-5ce4-48ed-873c-1e2f845f6b2b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:27:47.059095 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: WARNING py.warnings [None req-7d9eb8a1-5ce4-48ed-873c-1e2f845f6b2b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 09:27:47.059095 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: warnings.warn(msg) Aug 24 09:27:47.059095 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: Aug 24 09:27:47.059746 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9eb8a1-5ce4-48ed-873c-1e2f845f6b2b admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:27:47.059851 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f692dc-0f81-414d-8937-b1e6ca2e6ef5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:27:47.060317 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f692dc-0f81-414d-8937-b1e6ca2e6ef5 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:27:47.060610 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f692dc-0f81-414d-8937-b1e6ca2e6ef5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:27:47.060878 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f692dc-0f81-414d-8937-b1e6ca2e6ef5 admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:27:47.061498 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f692dc-0f81-414d-8937-b1e6ca2e6ef5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:27:47.063392 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f692dc-0f81-414d-8937-b1e6ca2e6ef5 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:27:47.143858 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 783/1563] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 24 09:27:47 2022] POST /identity/v3/users => generated 291 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:27:47.144772 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9eb8a1-5ce4-48ed-873c-1e2f845f6b2b admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:27:47.155198 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9eb8a1-5ce4-48ed-873c-1e2f845f6b2b admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:27:47.159966 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90146a1e-b6b8-4edf-87af-b2fb0646870b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:27:47.160562 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90146a1e-b6b8-4edf-87af-b2fb0646870b admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:27:47.160849 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90146a1e-b6b8-4edf-87af-b2fb0646870b admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:27:47.161124 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90146a1e-b6b8-4edf-87af-b2fb0646870b admin admin] PATH_INFO: `/v3/domains` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:27:47.161763 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90146a1e-b6b8-4edf-87af-b2fb0646870b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:27:47.162073 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90146a1e-b6b8-4edf-87af-b2fb0646870b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:27:47.163823 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90146a1e-b6b8-4edf-87af-b2fb0646870b admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:27:47.169918 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 781/1564] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 24 09:27:47 2022] POST /identity/v3/users => generated 287 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:27:47.169918 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f692dc-0f81-414d-8937-b1e6ca2e6ef5 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:27:47.183080 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 784/1565] 10.222.0.16 () {58 vars in 1153 bytes} [Wed Aug 24 09:27:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:27:47.183080 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90146a1e-b6b8-4edf-87af-b2fb0646870b admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:27:47.183347 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f692dc-0f81-414d-8937-b1e6ca2e6ef5 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:27:47.186327 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfb03d5-e6b2-4ce1-b00f-f5fa260f0122 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:27:47.186962 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfb03d5-e6b2-4ce1-b00f-f5fa260f0122 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:27:47.187251 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfb03d5-e6b2-4ce1-b00f-f5fa260f0122 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:27:47.187524 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfb03d5-e6b2-4ce1-b00f-f5fa260f0122 admin admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:27:47.188185 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfb03d5-e6b2-4ce1-b00f-f5fa260f0122 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:27:47.190085 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfb03d5-e6b2-4ce1-b00f-f5fa260f0122 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:27:47.192997 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90146a1e-b6b8-4edf-87af-b2fb0646870b admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:27:47.197659 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1fa574-6938-4604-8f0b-3298589916ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:27:47.198251 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1fa574-6938-4604-8f0b-3298589916ec admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:27:47.198530 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1fa574-6938-4604-8f0b-3298589916ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:27:47.198801 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1fa574-6938-4604-8f0b-3298589916ec admin admin] PATH_INFO: `/v3/domains` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:27:47.199430 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1fa574-6938-4604-8f0b-3298589916ec admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:27:47.199764 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1fa574-6938-4604-8f0b-3298589916ec admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:27:47.201929 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1fa574-6938-4604-8f0b-3298589916ec admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:27:47.213806 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 785/1566] 10.222.0.16 () {58 vars in 1153 bytes} [Wed Aug 24 09:27:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:27:47.214903 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1fa574-6938-4604-8f0b-3298589916ec admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:27:47.226362 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1fa574-6938-4604-8f0b-3298589916ec admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:27:47.230870 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f65b02a-b67e-4b10-b5a6-9ce721f8695d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:27:47.232082 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f65b02a-b67e-4b10-b5a6-9ce721f8695d admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:27:47.232407 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f65b02a-b67e-4b10-b5a6-9ce721f8695d admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:27:47.232675 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f65b02a-b67e-4b10-b5a6-9ce721f8695d admin admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:27:47.233286 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f65b02a-b67e-4b10-b5a6-9ce721f8695d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:27:47.235149 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f65b02a-b67e-4b10-b5a6-9ce721f8695d admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:27:47.251337 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:27:47.281471 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 782/1567] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 24 09:27:47 2022] POST /identity/v3/projects => generated 373 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:27:47.282290 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfb03d5-e6b2-4ce1-b00f-f5fa260f0122 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:27:47.304182 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:27:47.331364 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 786/1568] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 24 09:27:47 2022] POST /identity/v3/projects => generated 371 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:27:47.332310 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f65b02a-b67e-4b10-b5a6-9ce721f8695d admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:27:47.347285 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfb03d5-e6b2-4ce1-b00f-f5fa260f0122 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:27:47.397115 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f65b02a-b67e-4b10-b5a6-9ce721f8695d admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:27:47.401817 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6671940-ab6f-4815-8613-2ad3219396bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:27:47.402418 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6671940-ab6f-4815-8613-2ad3219396bb admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:27:47.402697 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6671940-ab6f-4815-8613-2ad3219396bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:27:47.402965 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6671940-ab6f-4815-8613-2ad3219396bb admin admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:27:47.403063 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646e39c0-312b-4420-8042-32b974abbf07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:27:47.403638 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6671940-ab6f-4815-8613-2ad3219396bb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:27:47.403757 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646e39c0-312b-4420-8042-32b974abbf07 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:27:47.403988 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646e39c0-312b-4420-8042-32b974abbf07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:27:47.404525 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646e39c0-312b-4420-8042-32b974abbf07 admin admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:27:47.405414 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646e39c0-312b-4420-8042-32b974abbf07 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:27:47.405696 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6671940-ab6f-4815-8613-2ad3219396bb admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:27:47.406311 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646e39c0-312b-4420-8042-32b974abbf07 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:27:47.408051 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646e39c0-312b-4420-8042-32b974abbf07 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:27:47.420492 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 783/1569] 10.222.0.16 () {58 vars in 1153 bytes} [Wed Aug 24 09:27:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:27:47.421671 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646e39c0-312b-4420-8042-32b974abbf07 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:27:47.431313 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646e39c0-312b-4420-8042-32b974abbf07 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:27:47.436607 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7961cf6b-0152-49b4-9d06-f64f25dcfa86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:27:47.440657 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7961cf6b-0152-49b4-9d06-f64f25dcfa86 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:27:47.440657 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7961cf6b-0152-49b4-9d06-f64f25dcfa86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:27:47.440657 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7961cf6b-0152-49b4-9d06-f64f25dcfa86 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:27:47.440657 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7961cf6b-0152-49b4-9d06-f64f25dcfa86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:27:47.440657 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7961cf6b-0152-49b4-9d06-f64f25dcfa86 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:27:47.450030 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 784/1570] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 24 09:27:47 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:27:47.453270 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7961cf6b-0152-49b4-9d06-f64f25dcfa86 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:27:47.460273 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7961cf6b-0152-49b4-9d06-f64f25dcfa86 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:27:47.465076 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2a8699-86b7-405a-a767-afe80104ab5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:27:47.465623 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2a8699-86b7-405a-a767-afe80104ab5f admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:27:47.466423 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2a8699-86b7-405a-a767-afe80104ab5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:27:47.466723 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2a8699-86b7-405a-a767-afe80104ab5f admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:27:47.467336 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2a8699-86b7-405a-a767-afe80104ab5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:27:47.468907 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2a8699-86b7-405a-a767-afe80104ab5f admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:27:47.475335 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 787/1571] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 24 09:27:47 2022] POST /identity/v3/projects => generated 389 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:27:47.476195 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6671940-ab6f-4815-8613-2ad3219396bb admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:27:47.480001 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 785/1572] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 24 09:27:47 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:27:47.480951 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2a8699-86b7-405a-a767-afe80104ab5f admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:27:47.492661 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2a8699-86b7-405a-a767-afe80104ab5f admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:27:47.543614 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6671940-ab6f-4815-8613-2ad3219396bb admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:27:47.549036 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1acb62-4f1c-48b7-855c-f5400797d39c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:27:47.549509 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c10436-32b2-4de8-8241-32b0babb3f6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:27:47.549885 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1acb62-4f1c-48b7-855c-f5400797d39c admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:27:47.550108 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c10436-32b2-4de8-8241-32b0babb3f6c admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:27:47.550348 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1acb62-4f1c-48b7-855c-f5400797d39c admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:27:47.550458 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c10436-32b2-4de8-8241-32b0babb3f6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:27:47.550643 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c10436-32b2-4de8-8241-32b0babb3f6c admin admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:27:47.550873 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1acb62-4f1c-48b7-855c-f5400797d39c admin admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:27:47.551254 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c10436-32b2-4de8-8241-32b0babb3f6c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:27:47.551710 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1acb62-4f1c-48b7-855c-f5400797d39c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:27:47.553190 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c10436-32b2-4de8-8241-32b0babb3f6c admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:27:47.553807 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1acb62-4f1c-48b7-855c-f5400797d39c admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:27:47.620672 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 786/1573] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 24 09:27:47 2022] POST /identity/v3/projects => generated 371 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:27:47.620672 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c10436-32b2-4de8-8241-32b0babb3f6c admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:27:47.630709 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 788/1574] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 24 09:27:47 2022] POST /identity/v3/projects => generated 379 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:27:47.630709 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1acb62-4f1c-48b7-855c-f5400797d39c admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:27:47.645042 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:27:47.680633 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c10436-32b2-4de8-8241-32b0babb3f6c admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:27:47.693085 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1acb62-4f1c-48b7-855c-f5400797d39c admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:27:47.698035 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476d8a61-cc46-460b-92c8-097bb9fc91ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:27:47.698645 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476d8a61-cc46-460b-92c8-097bb9fc91ee admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:27:47.698929 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476d8a61-cc46-460b-92c8-097bb9fc91ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:27:47.699207 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476d8a61-cc46-460b-92c8-097bb9fc91ee admin admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:27:47.699993 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476d8a61-cc46-460b-92c8-097bb9fc91ee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:27:47.702509 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476d8a61-cc46-460b-92c8-097bb9fc91ee admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:27:47.713258 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:27:47.737107 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b892bb4c-1c68-4eca-924c-291fca50f79c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:27:47.737719 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b892bb4c-1c68-4eca-924c-291fca50f79c admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:27:47.737999 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b892bb4c-1c68-4eca-924c-291fca50f79c admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:27:47.738271 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b892bb4c-1c68-4eca-924c-291fca50f79c admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:27:47.738915 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b892bb4c-1c68-4eca-924c-291fca50f79c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:27:47.741069 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b892bb4c-1c68-4eca-924c-291fca50f79c admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:27:47.810891 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 789/1575] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 24 09:27:47 2022] POST /identity/v3/users => generated 286 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:27:47.812002 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476d8a61-cc46-460b-92c8-097bb9fc91ee admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:27:47.820100 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 787/1576] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 24 09:27:47 2022] POST /identity/v3/users => generated 287 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:27:47.825755 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b892bb4c-1c68-4eca-924c-291fca50f79c admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:27:47.825962 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476d8a61-cc46-460b-92c8-097bb9fc91ee admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:27:47.829012 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc71f08-c026-4411-8b8d-9e6d97edcacd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:27:47.829622 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc71f08-c026-4411-8b8d-9e6d97edcacd admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:27:47.829906 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc71f08-c026-4411-8b8d-9e6d97edcacd admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:27:47.830184 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc71f08-c026-4411-8b8d-9e6d97edcacd admin admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:27:47.830807 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc71f08-c026-4411-8b8d-9e6d97edcacd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:27:47.831554 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b892bb4c-1c68-4eca-924c-291fca50f79c admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:27:47.832822 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc71f08-c026-4411-8b8d-9e6d97edcacd admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:27:47.836522 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381aeb51-f549-4691-9475-e5593bb13ce5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:27:47.837395 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381aeb51-f549-4691-9475-e5593bb13ce5 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:27:47.837895 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381aeb51-f549-4691-9475-e5593bb13ce5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:27:47.838173 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381aeb51-f549-4691-9475-e5593bb13ce5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:27:47.838871 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381aeb51-f549-4691-9475-e5593bb13ce5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:27:47.840538 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381aeb51-f549-4691-9475-e5593bb13ce5 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:27:47.851447 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 788/1577] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 24 09:27:47 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:27:47.852494 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381aeb51-f549-4691-9475-e5593bb13ce5 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:27:47.862101 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381aeb51-f549-4691-9475-e5593bb13ce5 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:27:47.866672 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aeda76e-44c2-4984-b5f6-8e165dd9dbe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:27:47.867251 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aeda76e-44c2-4984-b5f6-8e165dd9dbe5 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:27:47.867534 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aeda76e-44c2-4984-b5f6-8e165dd9dbe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:27:47.867842 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aeda76e-44c2-4984-b5f6-8e165dd9dbe5 admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:27:47.868450 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aeda76e-44c2-4984-b5f6-8e165dd9dbe5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:27:47.870346 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aeda76e-44c2-4984-b5f6-8e165dd9dbe5 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:27:47.915495 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 790/1578] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 24 09:27:47 2022] POST /identity/v3/projects => generated 375 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:27:47.918850 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc71f08-c026-4411-8b8d-9e6d97edcacd admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:27:47.942819 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 789/1579] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 24 09:27:47 2022] POST /identity/v3/users => generated 295 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:27:47.943929 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aeda76e-44c2-4984-b5f6-8e165dd9dbe5 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:27:47.954218 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aeda76e-44c2-4984-b5f6-8e165dd9dbe5 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:27:47.983673 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc71f08-c026-4411-8b8d-9e6d97edcacd admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:27:47.988332 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf775a4-b98b-466b-851f-0161f724bd83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:27:47.988934 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf775a4-b98b-466b-851f-0161f724bd83 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:27:47.989215 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf775a4-b98b-466b-851f-0161f724bd83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:27:47.989486 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf775a4-b98b-466b-851f-0161f724bd83 admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:27:47.990123 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf775a4-b98b-466b-851f-0161f724bd83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:27:47.991745 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf775a4-b98b-466b-851f-0161f724bd83 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:27:48.001402 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 791/1580] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 24 09:27:47 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:27:48.002248 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf775a4-b98b-466b-851f-0161f724bd83 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:27:48.012108 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf775a4-b98b-466b-851f-0161f724bd83 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:27:48.012583 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ce3081-9cbd-4769-98f0-43518af5273d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:27:48.013757 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ce3081-9cbd-4769-98f0-43518af5273d admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:27:48.014053 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ce3081-9cbd-4769-98f0-43518af5273d admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:27:48.014328 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ce3081-9cbd-4769-98f0-43518af5273d admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:27:48.014955 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ce3081-9cbd-4769-98f0-43518af5273d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:27:48.016958 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ce3081-9cbd-4769-98f0-43518af5273d admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:27:48.017278 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8f4844-3216-4451-916c-f500c5e0fee4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:27:48.017863 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8f4844-3216-4451-916c-f500c5e0fee4 admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:27:48.018146 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8f4844-3216-4451-916c-f500c5e0fee4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:27:48.018422 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8f4844-3216-4451-916c-f500c5e0fee4 admin admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:27:48.019315 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8f4844-3216-4451-916c-f500c5e0fee4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:27:48.021251 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8f4844-3216-4451-916c-f500c5e0fee4 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:27:48.160842 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 790/1581] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 24 09:27:47 2022] POST /identity/v3/users => generated 286 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:27:48.161841 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ce3081-9cbd-4769-98f0-43518af5273d admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:27:48.172439 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ce3081-9cbd-4769-98f0-43518af5273d admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:27:48.177754 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33054b68-165a-422e-bab3-20110566936c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:27:48.178342 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33054b68-165a-422e-bab3-20110566936c admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:27:48.178618 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33054b68-165a-422e-bab3-20110566936c admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:27:48.178888 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33054b68-165a-422e-bab3-20110566936c admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:27:48.179522 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33054b68-165a-422e-bab3-20110566936c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:27:48.181129 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33054b68-165a-422e-bab3-20110566936c admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:27:48.189364 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 792/1582] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 24 09:27:48 2022] POST /identity/v3/users => generated 290 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:27:48.190286 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8f4844-3216-4451-916c-f500c5e0fee4 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:27:48.192762 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 791/1583] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 24 09:27:48 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:27:48.193459 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33054b68-165a-422e-bab3-20110566936c admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:27:48.200822 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8f4844-3216-4451-916c-f500c5e0fee4 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:27:48.202661 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33054b68-165a-422e-bab3-20110566936c admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:27:48.205585 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593a7765-e264-4710-a45d-6138d41bff1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:27:48.206190 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593a7765-e264-4710-a45d-6138d41bff1a admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:27:48.206474 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593a7765-e264-4710-a45d-6138d41bff1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:27:48.206748 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593a7765-e264-4710-a45d-6138d41bff1a admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:27:48.207362 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f92a16-91fa-4dd3-81f7-226e3abcbfd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:27:48.207498 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593a7765-e264-4710-a45d-6138d41bff1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:27:48.208015 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f92a16-91fa-4dd3-81f7-226e3abcbfd3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:27:48.208290 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f92a16-91fa-4dd3-81f7-226e3abcbfd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:27:48.210150 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f92a16-91fa-4dd3-81f7-226e3abcbfd3 admin admin] PATH_INFO: `/v3/projects/a99aef25597a4ab480625b53c48cfe80/users/def04335e57e48409a6943fa231d47c3/roles/136cdf78e00a426997dd4e37faff489b` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:27:48.210255 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593a7765-e264-4710-a45d-6138d41bff1a admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:27:48.218352 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=91,keystone:INSERT=9 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:27:48.220873 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 793/1584] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 24 09:27:48 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:27:48.221987 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593a7765-e264-4710-a45d-6138d41bff1a admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:27:48.225392 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f92a16-91fa-4dd3-81f7-226e3abcbfd3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a99aef25597a4ab480625b53c48cfe80, user_id=def04335e57e48409a6943fa231d47c3, role_id=136cdf78e00a426997dd4e37faff489b)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:27:48.226428 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: WARNING py.warnings [None req-71f92a16-91fa-4dd3-81f7-226e3abcbfd3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 09:27:48.226428 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: warnings.warn(msg) Aug 24 09:27:48.226428 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: Aug 24 09:27:48.228731 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f92a16-91fa-4dd3-81f7-226e3abcbfd3 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:27:48.232141 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593a7765-e264-4710-a45d-6138d41bff1a admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:27:48.236814 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676ae6d9-b9d2-4d03-b93b-95ced3feb93d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:27:48.237419 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676ae6d9-b9d2-4d03-b93b-95ced3feb93d admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:27:48.237697 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676ae6d9-b9d2-4d03-b93b-95ced3feb93d admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:27:48.237967 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676ae6d9-b9d2-4d03-b93b-95ced3feb93d admin admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:27:48.238605 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676ae6d9-b9d2-4d03-b93b-95ced3feb93d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:27:48.240544 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676ae6d9-b9d2-4d03-b93b-95ced3feb93d admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:27:48.308462 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 792/1585] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 24 09:27:48 2022] PUT /identity/v3/projects/a99aef25597a4ab480625b53c48cfe80/users/def04335e57e48409a6943fa231d47c3/roles/136cdf78e00a426997dd4e37faff489b => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:27:48.309327 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f92a16-91fa-4dd3-81f7-226e3abcbfd3 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:27:48.323226 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 794/1586] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 24 09:27:48 2022] POST /identity/v3/users => generated 288 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:27:48.323226 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676ae6d9-b9d2-4d03-b93b-95ced3feb93d admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:27:48.331041 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676ae6d9-b9d2-4d03-b93b-95ced3feb93d admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:27:48.380334 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f92a16-91fa-4dd3-81f7-226e3abcbfd3 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:27:48.385054 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55509182-f3c9-47f5-bd18-15496c88330f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:27:48.385649 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55509182-f3c9-47f5-bd18-15496c88330f admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:27:48.385924 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55509182-f3c9-47f5-bd18-15496c88330f admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:27:48.386193 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55509182-f3c9-47f5-bd18-15496c88330f admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:27:48.386826 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55509182-f3c9-47f5-bd18-15496c88330f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:27:48.387536 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c9997c-f966-4faa-a93f-be520ae353ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:27:48.388242 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c9997c-f966-4faa-a93f-be520ae353ed admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:27:48.388433 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55509182-f3c9-47f5-bd18-15496c88330f admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:27:48.388542 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c9997c-f966-4faa-a93f-be520ae353ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:27:48.388781 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c9997c-f966-4faa-a93f-be520ae353ed admin admin] PATH_INFO: `/v3/projects/0d04a503a13a48439073f044775b24a1/users/b3d64b436fc8454087a68c0506264de6/roles/136cdf78e00a426997dd4e37faff489b` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:27:48.397932 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 793/1587] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 24 09:27:48 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:27:48.398724 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55509182-f3c9-47f5-bd18-15496c88330f admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:27:48.407139 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c9997c-f966-4faa-a93f-be520ae353ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d04a503a13a48439073f044775b24a1, user_id=b3d64b436fc8454087a68c0506264de6, role_id=136cdf78e00a426997dd4e37faff489b)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:27:48.408309 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55509182-f3c9-47f5-bd18-15496c88330f admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:27:48.409951 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c9997c-f966-4faa-a93f-be520ae353ed admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:27:48.413010 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bf0f25-b9b2-431a-bc78-80a2a26968e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:27:48.413526 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bf0f25-b9b2-431a-bc78-80a2a26968e5 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:27:48.413804 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bf0f25-b9b2-431a-bc78-80a2a26968e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:27:48.414075 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bf0f25-b9b2-431a-bc78-80a2a26968e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:27:48.414699 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36bf0f25-b9b2-431a-bc78-80a2a26968e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:27:48.416295 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36bf0f25-b9b2-431a-bc78-80a2a26968e5 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:27:48.426509 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 794/1588] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 24 09:27:48 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:27:48.427345 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bf0f25-b9b2-431a-bc78-80a2a26968e5 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:27:48.436910 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bf0f25-b9b2-431a-bc78-80a2a26968e5 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:27:48.441387 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1846fb-25ff-4eba-96c5-077e3a095e81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:27:48.442357 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1846fb-25ff-4eba-96c5-077e3a095e81 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:27:48.442357 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1846fb-25ff-4eba-96c5-077e3a095e81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:27:48.442357 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1846fb-25ff-4eba-96c5-077e3a095e81 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:27:48.443033 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1846fb-25ff-4eba-96c5-077e3a095e81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:27:48.444648 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1846fb-25ff-4eba-96c5-077e3a095e81 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:27:48.455027 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 795/1589] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 24 09:27:48 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:27:48.455798 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1846fb-25ff-4eba-96c5-077e3a095e81 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:27:48.464955 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1846fb-25ff-4eba-96c5-077e3a095e81 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:27:48.469993 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2a4f41-3d32-488c-be7f-40ad69be963e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:27:48.470536 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2a4f41-3d32-488c-be7f-40ad69be963e admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:27:48.470812 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2a4f41-3d32-488c-be7f-40ad69be963e admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:27:48.471083 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2a4f41-3d32-488c-be7f-40ad69be963e admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:27:48.471720 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2a4f41-3d32-488c-be7f-40ad69be963e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:27:48.473240 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2a4f41-3d32-488c-be7f-40ad69be963e admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:27:48.475710 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 795/1590] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 24 09:27:48 2022] PUT /identity/v3/projects/0d04a503a13a48439073f044775b24a1/users/b3d64b436fc8454087a68c0506264de6/roles/136cdf78e00a426997dd4e37faff489b => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:27:48.476540 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c9997c-f966-4faa-a93f-be520ae353ed admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:27:48.481921 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:INSERT=9,keystone:SELECT=91 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:27:48.487025 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 796/1591] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 24 09:27:48 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:27:48.487025 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6602d71f-4f57-464b-a425-496802a5fce1 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:27:48.487025 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6602d71f-4f57-464b-a425-496802a5fce1 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:27:48.487025 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6602d71f-4f57-464b-a425-496802a5fce1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:27:48.504854 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-6602d71f-4f57-464b-a425-496802a5fce1 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:27:48.506117 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-6602d71f-4f57-464b-a425-496802a5fce1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:27:48.549651 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c9997c-f966-4faa-a93f-be520ae353ed admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:27:48.554279 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695b5122-dc45-47bf-b7e5-7a1c377405d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:27:48.554883 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695b5122-dc45-47bf-b7e5-7a1c377405d4 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:27:48.555166 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695b5122-dc45-47bf-b7e5-7a1c377405d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:27:48.555436 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695b5122-dc45-47bf-b7e5-7a1c377405d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:27:48.556100 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695b5122-dc45-47bf-b7e5-7a1c377405d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:27:48.557692 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695b5122-dc45-47bf-b7e5-7a1c377405d4 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:27:48.565223 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 797/1592] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 09:27:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:27:48.566064 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6602d71f-4f57-464b-a425-496802a5fce1 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:27:48.567345 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 796/1593] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 24 09:27:48 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:27:48.568181 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695b5122-dc45-47bf-b7e5-7a1c377405d4 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:27:48.578917 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695b5122-dc45-47bf-b7e5-7a1c377405d4 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:27:48.579187 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6602d71f-4f57-464b-a425-496802a5fce1 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:27:48.583548 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d2ee86-942e-4df7-813f-7e00074e5621 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:27:48.583711 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0894c5ad-7dc2-4f52-8b19-f1ec5ce351ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:27:48.584171 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d2ee86-942e-4df7-813f-7e00074e5621 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:27:48.584281 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0894c5ad-7dc2-4f52-8b19-f1ec5ce351ec admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:27:48.584450 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d2ee86-942e-4df7-813f-7e00074e5621 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:27:48.584549 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0894c5ad-7dc2-4f52-8b19-f1ec5ce351ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:27:48.584722 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d2ee86-942e-4df7-813f-7e00074e5621 admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:27:48.584827 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0894c5ad-7dc2-4f52-8b19-f1ec5ce351ec admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:27:48.585352 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d2ee86-942e-4df7-813f-7e00074e5621 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:27:48.585460 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0894c5ad-7dc2-4f52-8b19-f1ec5ce351ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:27:48.587184 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d2ee86-942e-4df7-813f-7e00074e5621 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:27:48.587288 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0894c5ad-7dc2-4f52-8b19-f1ec5ce351ec admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:27:48.600897 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 798/1594] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 24 09:27:48 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:27:48.601085 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 797/1595] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 24 09:27:48 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:27:48.601222 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0894c5ad-7dc2-4f52-8b19-f1ec5ce351ec admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:27:48.601311 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d2ee86-942e-4df7-813f-7e00074e5621 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:27:48.609960 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d2ee86-942e-4df7-813f-7e00074e5621 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:27:48.611328 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0894c5ad-7dc2-4f52-8b19-f1ec5ce351ec admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:27:48.614546 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935e9d76-e67c-4304-b8e0-13b92f8cdb8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:27:48.615134 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935e9d76-e67c-4304-b8e0-13b92f8cdb8f admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:27:48.615414 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935e9d76-e67c-4304-b8e0-13b92f8cdb8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:27:48.615719 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935e9d76-e67c-4304-b8e0-13b92f8cdb8f admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:27:48.616145 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c62a17c-c53f-43c8-bddf-85162e32f8cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:27:48.616407 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935e9d76-e67c-4304-b8e0-13b92f8cdb8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:27:48.616714 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c62a17c-c53f-43c8-bddf-85162e32f8cd admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:27:48.616995 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c62a17c-c53f-43c8-bddf-85162e32f8cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:27:48.617275 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c62a17c-c53f-43c8-bddf-85162e32f8cd admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:27:48.617907 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c62a17c-c53f-43c8-bddf-85162e32f8cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:27:48.618019 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935e9d76-e67c-4304-b8e0-13b92f8cdb8f admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:27:48.619478 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c62a17c-c53f-43c8-bddf-85162e32f8cd admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:27:48.628438 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 798/1596] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 24 09:27:48 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:27:48.628438 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935e9d76-e67c-4304-b8e0-13b92f8cdb8f admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:27:48.631994 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 799/1597] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 24 09:27:48 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:27:48.631994 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06388f1-1217-4a0c-9c63-983eff6a6daa None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:27:48.631994 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06388f1-1217-4a0c-9c63-983eff6a6daa None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:27:48.631994 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06388f1-1217-4a0c-9c63-983eff6a6daa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:27:48.643390 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935e9d76-e67c-4304-b8e0-13b92f8cdb8f admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:27:48.648720 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2f5f64-d7b2-43af-9a77-37d17fa31dab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:27:48.649386 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2f5f64-d7b2-43af-9a77-37d17fa31dab admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:27:48.649672 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2f5f64-d7b2-43af-9a77-37d17fa31dab admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:27:48.649950 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2f5f64-d7b2-43af-9a77-37d17fa31dab admin admin] PATH_INFO: `/v3/projects/1a132169db34442cb6e7f2ba7cf94962/users/a99817614d86407ea10902774e47fca2/roles/136cdf78e00a426997dd4e37faff489b` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:27:48.661713 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-c06388f1-1217-4a0c-9c63-983eff6a6daa None None] MFA Rules not processed for user `b3d64b436fc8454087a68c0506264de6`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:27:48.662947 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-c06388f1-1217-4a0c-9c63-983eff6a6daa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:27:48.667205 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2f5f64-d7b2-43af-9a77-37d17fa31dab admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a132169db34442cb6e7f2ba7cf94962, user_id=a99817614d86407ea10902774e47fca2, role_id=136cdf78e00a426997dd4e37faff489b)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:27:48.669868 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2f5f64-d7b2-43af-9a77-37d17fa31dab admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:27:48.680932 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:27:48.728226 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 800/1598] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 09:27:48 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:27:48.729252 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06388f1-1217-4a0c-9c63-983eff6a6daa None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:27:48.737541 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 799/1599] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 24 09:27:48 2022] PUT /identity/v3/projects/1a132169db34442cb6e7f2ba7cf94962/users/a99817614d86407ea10902774e47fca2/roles/136cdf78e00a426997dd4e37faff489b => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:27:48.745631 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2f5f64-d7b2-43af-9a77-37d17fa31dab admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:27:48.745780 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06388f1-1217-4a0c-9c63-983eff6a6daa None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:27:48.798224 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2f5f64-d7b2-43af-9a77-37d17fa31dab admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:27:48.801954 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764c1213-ada5-4a8d-a206-3038fe6f9aab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e5260e798404e8bb12b25ca8ea97052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:27:48.802566 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764c1213-ada5-4a8d-a206-3038fe6f9aab service neutron] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:27:48.802986 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764c1213-ada5-4a8d-a206-3038fe6f9aab service neutron] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:27:48.803134 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764c1213-ada5-4a8d-a206-3038fe6f9aab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:27:48.810303 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31369bad-9cb8-4c6d-8fc7-c63a19926421 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:27:48.810993 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31369bad-9cb8-4c6d-8fc7-c63a19926421 admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:27:48.811283 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31369bad-9cb8-4c6d-8fc7-c63a19926421 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:27:48.811669 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31369bad-9cb8-4c6d-8fc7-c63a19926421 admin admin] PATH_INFO: `/v3/projects/803ce13612e0460682ef8b4d42781a99/users/b9099dc3ad92431cbac98f144690038c/roles/136cdf78e00a426997dd4e37faff489b` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:27:48.825892 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31369bad-9cb8-4c6d-8fc7-c63a19926421 admin admin] RBAC: Authorizing `identity:create_grant(project_id=803ce13612e0460682ef8b4d42781a99, user_id=b9099dc3ad92431cbac98f144690038c, role_id=136cdf78e00a426997dd4e37faff489b)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:27:48.829173 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31369bad-9cb8-4c6d-8fc7-c63a19926421 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:27:48.864290 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764c1213-ada5-4a8d-a206-3038fe6f9aab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:27:48.865970 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764c1213-ada5-4a8d-a206-3038fe6f9aab service neutron] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:27:48.876262 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 800/1600] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:27:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:27:48.877047 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764c1213-ada5-4a8d-a206-3038fe6f9aab service neutron] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:27:48.882527 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:27:48.886488 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764c1213-ada5-4a8d-a206-3038fe6f9aab service neutron] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:27:48.896575 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 801/1601] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 24 09:27:48 2022] PUT /identity/v3/projects/803ce13612e0460682ef8b4d42781a99/users/b9099dc3ad92431cbac98f144690038c/roles/136cdf78e00a426997dd4e37faff489b => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:27:48.897500 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31369bad-9cb8-4c6d-8fc7-c63a19926421 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:27:48.945328 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df9a35c-f105-43f4-ba65-ebe6c55d0da3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:27:48.946003 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df9a35c-f105-43f4-ba65-ebe6c55d0da3 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:27:48.946286 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df9a35c-f105-43f4-ba65-ebe6c55d0da3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:27:48.946636 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df9a35c-f105-43f4-ba65-ebe6c55d0da3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:27:48.947278 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df9a35c-f105-43f4-ba65-ebe6c55d0da3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:27:48.948905 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df9a35c-f105-43f4-ba65-ebe6c55d0da3 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:27:48.959147 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 801/1602] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 24 09:27:48 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:27:48.959781 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df9a35c-f105-43f4-ba65-ebe6c55d0da3 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:27:48.963700 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31369bad-9cb8-4c6d-8fc7-c63a19926421 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:27:48.968439 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bd8d6b-b8dc-4e52-9554-860cc4dca864 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:27:48.969171 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df9a35c-f105-43f4-ba65-ebe6c55d0da3 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:27:48.969286 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bd8d6b-b8dc-4e52-9554-860cc4dca864 admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:27:48.969549 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bd8d6b-b8dc-4e52-9554-860cc4dca864 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:27:48.969829 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bd8d6b-b8dc-4e52-9554-860cc4dca864 admin admin] PATH_INFO: `/v3/projects/d00ce4a2669e43dcb5bc8a2d77b11c2f/users/cbcaa84bc9c54d7aa4178d0174a5936e/roles/136cdf78e00a426997dd4e37faff489b` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:27:48.973764 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21da252-178a-40e5-8f52-2afd735a1643 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:27:48.974376 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21da252-178a-40e5-8f52-2afd735a1643 admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:27:48.974649 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21da252-178a-40e5-8f52-2afd735a1643 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:27:48.974916 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21da252-178a-40e5-8f52-2afd735a1643 admin admin] PATH_INFO: `/v3/projects/fe8949d7fc8b4ca89fd81421d7e94122/users/6cad69b6035948fe91e9beb66c807d28/roles/136cdf78e00a426997dd4e37faff489b` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:27:48.986100 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bd8d6b-b8dc-4e52-9554-860cc4dca864 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d00ce4a2669e43dcb5bc8a2d77b11c2f, user_id=cbcaa84bc9c54d7aa4178d0174a5936e, role_id=136cdf78e00a426997dd4e37faff489b)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:27:48.988928 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bd8d6b-b8dc-4e52-9554-860cc4dca864 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:27:48.989916 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21da252-178a-40e5-8f52-2afd735a1643 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe8949d7fc8b4ca89fd81421d7e94122, user_id=6cad69b6035948fe91e9beb66c807d28, role_id=136cdf78e00a426997dd4e37faff489b)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:27:48.992655 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21da252-178a-40e5-8f52-2afd735a1643 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:27:49.051956 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 802/1603] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 24 09:27:48 2022] PUT /identity/v3/projects/d00ce4a2669e43dcb5bc8a2d77b11c2f/users/cbcaa84bc9c54d7aa4178d0174a5936e/roles/136cdf78e00a426997dd4e37faff489b => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:27:49.057594 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bd8d6b-b8dc-4e52-9554-860cc4dca864 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:27:49.064186 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 802/1604] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 24 09:27:48 2022] PUT /identity/v3/projects/fe8949d7fc8b4ca89fd81421d7e94122/users/6cad69b6035948fe91e9beb66c807d28/roles/136cdf78e00a426997dd4e37faff489b => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:27:49.065294 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef73cd8-d2e5-4371-bb08-b892eefbfd0d None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:27:49.065713 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef73cd8-d2e5-4371-bb08-b892eefbfd0d None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:27:49.065987 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef73cd8-d2e5-4371-bb08-b892eefbfd0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:27:49.093855 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:27:49.095973 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-fef73cd8-d2e5-4371-bb08-b892eefbfd0d None None] MFA Rules not processed for user `a99817614d86407ea10902774e47fca2`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:27:49.097198 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-fef73cd8-d2e5-4371-bb08-b892eefbfd0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:27:49.128025 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bd8d6b-b8dc-4e52-9554-860cc4dca864 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:27:49.160797 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 803/1605] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 09:27:49 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:27:49.161973 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc68bfd4-5cda-483b-8a6f-7d43254054ec None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:27:49.162274 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc68bfd4-5cda-483b-8a6f-7d43254054ec None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:27:49.162556 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc68bfd4-5cda-483b-8a6f-7d43254054ec None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:27:49.180116 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 804/1606] 10.222.0.16 () {54 vars in 962 bytes} [Wed Aug 24 09:27:49 2022] GET /identity => generated 270 bytes in 19 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:27:49.182246 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8052598b-93fd-45da-b003-524aae90aefe None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:27:49.183289 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8052598b-93fd-45da-b003-524aae90aefe None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:27:49.183730 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8052598b-93fd-45da-b003-524aae90aefe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:27:49.189154 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63836caa-c205-4e2d-8b5f-4b6f945bdb38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:27:49.190185 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63836caa-c205-4e2d-8b5f-4b6f945bdb38 admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:27:49.190685 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63836caa-c205-4e2d-8b5f-4b6f945bdb38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:27:49.191115 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63836caa-c205-4e2d-8b5f-4b6f945bdb38 admin admin] PATH_INFO: `/v3/projects/05dbe88d418a40bea60335d5411e9d1e/users/862990dac5f84b07abc9fe26ccd6a41a/roles/136cdf78e00a426997dd4e37faff489b` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:27:49.212255 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-8052598b-93fd-45da-b003-524aae90aefe None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:27:49.212255 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-8052598b-93fd-45da-b003-524aae90aefe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:27:49.212480 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63836caa-c205-4e2d-8b5f-4b6f945bdb38 admin admin] RBAC: Authorizing `identity:create_grant(project_id=05dbe88d418a40bea60335d5411e9d1e, user_id=862990dac5f84b07abc9fe26ccd6a41a, role_id=136cdf78e00a426997dd4e37faff489b)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:27:49.214877 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63836caa-c205-4e2d-8b5f-4b6f945bdb38 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:27:49.222644 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 805/1607] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 09:27:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:27:49.222644 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8052598b-93fd-45da-b003-524aae90aefe None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:27:49.231338 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8052598b-93fd-45da-b003-524aae90aefe None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:27:49.239172 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f85d576-58c4-4745-81e9-ff5ebc845d94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:27:49.239885 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f85d576-58c4-4745-81e9-ff5ebc845d94 admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:27:49.240171 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f85d576-58c4-4745-81e9-ff5ebc845d94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:27:49.240451 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f85d576-58c4-4745-81e9-ff5ebc845d94 admin admin] PATH_INFO: `/v3/projects/b833ebb3692e4b51aceaba3d9da870bf/users/bc655ce402694d6589e848650ce88204/roles/136cdf78e00a426997dd4e37faff489b` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:27:49.258002 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f85d576-58c4-4745-81e9-ff5ebc845d94 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b833ebb3692e4b51aceaba3d9da870bf, user_id=bc655ce402694d6589e848650ce88204, role_id=136cdf78e00a426997dd4e37faff489b)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:27:49.260747 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f85d576-58c4-4745-81e9-ff5ebc845d94 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:27:49.274428 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 803/1608] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 24 09:27:49 2022] PUT /identity/v3/projects/05dbe88d418a40bea60335d5411e9d1e/users/862990dac5f84b07abc9fe26ccd6a41a/roles/136cdf78e00a426997dd4e37faff489b => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:27:49.275931 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80bc5c2-927b-4f98-812e-b2eb533eafa4 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:27:49.276223 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80bc5c2-927b-4f98-812e-b2eb533eafa4 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:27:49.276499 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80bc5c2-927b-4f98-812e-b2eb533eafa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:27:49.301869 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-a80bc5c2-927b-4f98-812e-b2eb533eafa4 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:27:49.303363 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-a80bc5c2-927b-4f98-812e-b2eb533eafa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:27:49.337709 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 806/1609] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 24 09:27:49 2022] PUT /identity/v3/projects/b833ebb3692e4b51aceaba3d9da870bf/users/bc655ce402694d6589e848650ce88204/roles/136cdf78e00a426997dd4e37faff489b => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:27:49.337709 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b619fa-f15d-4a20-877f-ac5909308c5b None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:27:49.337709 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b619fa-f15d-4a20-877f-ac5909308c5b None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:27:49.337709 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b619fa-f15d-4a20-877f-ac5909308c5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:27:49.355755 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-50b619fa-f15d-4a20-877f-ac5909308c5b None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:27:49.356966 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-50b619fa-f15d-4a20-877f-ac5909308c5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:27:49.363675 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 804/1610] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 09:27:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:27:49.364791 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b04008-f927-4530-8eea-34321651997e None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:27:49.365086 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b04008-f927-4530-8eea-34321651997e None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:27:49.365358 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b04008-f927-4530-8eea-34321651997e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:27:49.391017 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:27:49.406066 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-52b04008-f927-4530-8eea-34321651997e None None] MFA Rules not processed for user `1f055f7dc51d43bda9b315ca67f11d58`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:27:49.407281 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-52b04008-f927-4530-8eea-34321651997e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:27:49.426859 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 807/1611] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 09:27:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:27:49.426859 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ceb7b88-8eaa-4d21-963f-09075727e959 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:27:49.426859 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ceb7b88-8eaa-4d21-963f-09075727e959 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:27:49.426859 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ceb7b88-8eaa-4d21-963f-09075727e959 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:27:49.432471 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 808/1612] 10.222.0.16 () {54 vars in 962 bytes} [Wed Aug 24 09:27:49 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:27:49.432471 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ceb7b88-8eaa-4d21-963f-09075727e959 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:27:49.500171 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ceb7b88-8eaa-4d21-963f-09075727e959 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:27:49.504632 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e07dfcd-129b-4a30-9545-d2fda1c3d56d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e5260e798404e8bb12b25ca8ea97052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:27:49.505328 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e07dfcd-129b-4a30-9545-d2fda1c3d56d service neutron] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:27:49.505704 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e07dfcd-129b-4a30-9545-d2fda1c3d56d service neutron] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:27:49.506052 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e07dfcd-129b-4a30-9545-d2fda1c3d56d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:27:49.511284 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 805/1613] 10.222.0.16 () {58 vars in 1059 bytes} [Wed Aug 24 09:27:49 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:27:49.513752 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79d87f0-41c3-4de4-9ff8-f5b7f37581a1 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:27:49.513981 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79d87f0-41c3-4de4-9ff8-f5b7f37581a1 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:27:49.514388 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79d87f0-41c3-4de4-9ff8-f5b7f37581a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:27:49.517521 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e07dfcd-129b-4a30-9545-d2fda1c3d56d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:27:49.519195 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e07dfcd-129b-4a30-9545-d2fda1c3d56d service neutron] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:27:49.531793 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 809/1614] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:27:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:27:49.533150 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c522bb-2844-4d2a-bb78-60273a1b4c1e None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:27:49.533532 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c522bb-2844-4d2a-bb78-60273a1b4c1e None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:27:49.533907 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c522bb-2844-4d2a-bb78-60273a1b4c1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:27:49.547502 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-e79d87f0-41c3-4de4-9ff8-f5b7f37581a1 None None] MFA Rules not processed for user `862990dac5f84b07abc9fe26ccd6a41a`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:27:49.552828 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-e79d87f0-41c3-4de4-9ff8-f5b7f37581a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:27:49.567323 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-70c522bb-2844-4d2a-bb78-60273a1b4c1e None None] MFA Rules not processed for user `bc655ce402694d6589e848650ce88204`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:27:49.568669 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-70c522bb-2844-4d2a-bb78-60273a1b4c1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:27:49.629718 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:27:49.640292 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 810/1615] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 09:27:49 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:27:49.641102 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 806/1616] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 09:27:49 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:27:49.641760 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c522bb-2844-4d2a-bb78-60273a1b4c1e None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:27:49.642075 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79d87f0-41c3-4de4-9ff8-f5b7f37581a1 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:27:49.654486 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79d87f0-41c3-4de4-9ff8-f5b7f37581a1 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:27:49.655963 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c522bb-2844-4d2a-bb78-60273a1b4c1e None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:27:49.658856 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d9190b-6a4b-4467-8385-6bb6e6b4b786 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e5260e798404e8bb12b25ca8ea97052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:27:49.659466 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d9190b-6a4b-4467-8385-6bb6e6b4b786 service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:27:49.659791 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d9190b-6a4b-4467-8385-6bb6e6b4b786 service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:27:49.660069 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d9190b-6a4b-4467-8385-6bb6e6b4b786 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:27:49.660482 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a761d0-518c-4aec-92e7-25b4b060d833 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e5260e798404e8bb12b25ca8ea97052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:27:49.661400 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a761d0-518c-4aec-92e7-25b4b060d833 service neutron] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:27:49.661827 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a761d0-518c-4aec-92e7-25b4b060d833 service neutron] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:27:49.662248 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a761d0-518c-4aec-92e7-25b4b060d833 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:27:49.672639 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d9190b-6a4b-4467-8385-6bb6e6b4b786 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:27:49.674535 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d9190b-6a4b-4467-8385-6bb6e6b4b786 service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:27:49.677840 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a761d0-518c-4aec-92e7-25b4b060d833 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:27:49.679362 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a761d0-518c-4aec-92e7-25b4b060d833 service neutron] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:27:49.687621 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 807/1617] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:27:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:27:49.689048 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aada83f-9a68-47bf-8764-b6208ff467f7 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:27:49.689491 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aada83f-9a68-47bf-8764-b6208ff467f7 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:27:49.689834 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aada83f-9a68-47bf-8764-b6208ff467f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:27:49.692224 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 811/1618] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:27:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:27:49.692915 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a761d0-518c-4aec-92e7-25b4b060d833 service neutron] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:27:49.709222 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a761d0-518c-4aec-92e7-25b4b060d833 service neutron] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:27:49.711541 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-9aada83f-9a68-47bf-8764-b6208ff467f7 None None] MFA Rules not processed for user `1f055f7dc51d43bda9b315ca67f11d58`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:27:49.713279 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-9aada83f-9a68-47bf-8764-b6208ff467f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:27:49.715963 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f793cd9-227b-46b7-aa47-c41e58924772 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f055f7dc51d43bda9b315ca67f11d58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:27:49.716593 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f793cd9-227b-46b7-aa47-c41e58924772 service nova] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:27:49.717242 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f793cd9-227b-46b7-aa47-c41e58924772 service nova] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:27:49.717523 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f793cd9-227b-46b7-aa47-c41e58924772 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:27:49.727683 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 808/1619] 10.222.0.16 () {58 vars in 1059 bytes} [Wed Aug 24 09:27:49 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:27:49.728469 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aada83f-9a68-47bf-8764-b6208ff467f7 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:27:49.739230 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aada83f-9a68-47bf-8764-b6208ff467f7 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:27:49.749789 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aaab47e-1d32-4382-9685-1e67df48af12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:27:49.750622 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aaab47e-1d32-4382-9685-1e67df48af12 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:27:49.750976 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aaab47e-1d32-4382-9685-1e67df48af12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:27:49.751322 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aaab47e-1d32-4382-9685-1e67df48af12 admin admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:27:49.752053 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aaab47e-1d32-4382-9685-1e67df48af12 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:27:49.754122 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aaab47e-1d32-4382-9685-1e67df48af12 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:27:49.783460 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f793cd9-227b-46b7-aa47-c41e58924772 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:27:49.785767 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f793cd9-227b-46b7-aa47-c41e58924772 service nova] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:27:49.791563 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:27:49.798003 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 812/1620] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:27:49 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:27:49.798942 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f793cd9-227b-46b7-aa47-c41e58924772 service nova] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:27:49.809060 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f793cd9-227b-46b7-aa47-c41e58924772 service nova] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:27:49.817438 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ef00c9-47a4-408e-b4b8-095373a3152b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:27:49.818327 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ef00c9-47a4-408e-b4b8-095373a3152b admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:27:49.818787 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ef00c9-47a4-408e-b4b8-095373a3152b admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:27:49.819214 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ef00c9-47a4-408e-b4b8-095373a3152b admin admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:27:49.820088 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ef00c9-47a4-408e-b4b8-095373a3152b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:27:49.822201 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ef00c9-47a4-408e-b4b8-095373a3152b admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:27:49.852877 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 809/1621] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 24 09:27:49 2022] POST /identity/v3/projects => generated 377 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:27:49.853927 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aaab47e-1d32-4382-9685-1e67df48af12 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:27:49.940976 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 813/1622] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 24 09:27:49 2022] POST /identity/v3/projects => generated 373 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:27:49.942311 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ef00c9-47a4-408e-b4b8-095373a3152b admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:27:49.961748 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aaab47e-1d32-4382-9685-1e67df48af12 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:27:49.991681 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:27:50.077516 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9768fd0c-1c8e-4077-8cbf-411a4df74cab service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f055f7dc51d43bda9b315ca67f11d58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:27:50.080521 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9768fd0c-1c8e-4077-8cbf-411a4df74cab service nova] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:27:50.080521 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9768fd0c-1c8e-4077-8cbf-411a4df74cab service nova] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:27:50.080727 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9768fd0c-1c8e-4077-8cbf-411a4df74cab service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:27:50.085982 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ef00c9-47a4-408e-b4b8-095373a3152b admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:27:50.096236 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3543f5-ebe3-4a08-a8eb-a2e87df8964d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:27:50.097984 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3543f5-ebe3-4a08-a8eb-a2e87df8964d admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:27:50.098487 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3543f5-ebe3-4a08-a8eb-a2e87df8964d admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:27:50.099089 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3543f5-ebe3-4a08-a8eb-a2e87df8964d admin admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:27:50.100847 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3543f5-ebe3-4a08-a8eb-a2e87df8964d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:27:50.104860 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3543f5-ebe3-4a08-a8eb-a2e87df8964d admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:27:50.158121 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9768fd0c-1c8e-4077-8cbf-411a4df74cab service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:27:50.159893 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9768fd0c-1c8e-4077-8cbf-411a4df74cab service nova] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:27:50.169321 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 810/1623] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:27:49 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:27:50.170185 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9768fd0c-1c8e-4077-8cbf-411a4df74cab service nova] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:27:50.192371 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9768fd0c-1c8e-4077-8cbf-411a4df74cab service nova] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:27:50.195838 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 814/1624] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 24 09:27:49 2022] POST /identity/v3/users => generated 295 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:27:50.197358 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bba253-cab3-4b9b-968f-fa7ea3b5da86 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:27:50.197745 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bba253-cab3-4b9b-968f-fa7ea3b5da86 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:27:50.198145 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bba253-cab3-4b9b-968f-fa7ea3b5da86 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:27:50.203424 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c82ff8-0f76-4b97-a7a4-77db398616ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:27:50.204223 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c82ff8-0f76-4b97-a7a4-77db398616ad admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:27:50.206611 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c82ff8-0f76-4b97-a7a4-77db398616ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:27:50.206611 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c82ff8-0f76-4b97-a7a4-77db398616ad admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:27:50.206813 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 815/1625] 10.222.0.16 () {54 vars in 912 bytes} [Wed Aug 24 09:27:50 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:27:50.206903 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c82ff8-0f76-4b97-a7a4-77db398616ad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:27:50.211786 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bba253-cab3-4b9b-968f-fa7ea3b5da86 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:27:50.214722 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c82ff8-0f76-4b97-a7a4-77db398616ad admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:27:50.229592 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bba253-cab3-4b9b-968f-fa7ea3b5da86 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:27:50.235072 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78529d6-6ff7-42b4-babe-30b341b0ae72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:27:50.235862 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78529d6-6ff7-42b4-babe-30b341b0ae72 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:27:50.236237 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78529d6-6ff7-42b4-babe-30b341b0ae72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:27:50.236604 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78529d6-6ff7-42b4-babe-30b341b0ae72 admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:27:50.237357 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78529d6-6ff7-42b4-babe-30b341b0ae72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:27:50.239639 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78529d6-6ff7-42b4-babe-30b341b0ae72 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:27:50.258130 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 816/1626] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 24 09:27:50 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:27:50.259553 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca00e0e-3903-48db-b261-8734a9305506 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:27:50.259966 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca00e0e-3903-48db-b261-8734a9305506 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:27:50.260339 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca00e0e-3903-48db-b261-8734a9305506 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:27:50.289823 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-fca00e0e-3903-48db-b261-8734a9305506 None None] MFA Rules not processed for user `1f055f7dc51d43bda9b315ca67f11d58`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:27:50.291130 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-fca00e0e-3903-48db-b261-8734a9305506 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:27:50.306056 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 811/1627] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 24 09:27:50 2022] POST /identity/v3/users => generated 293 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:27:50.307064 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c82ff8-0f76-4b97-a7a4-77db398616ad admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:27:50.315420 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 817/1628] 10.222.0.16 () {58 vars in 1009 bytes} [Wed Aug 24 09:27:50 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:27:50.316895 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca00e0e-3903-48db-b261-8734a9305506 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:27:50.333636 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c82ff8-0f76-4b97-a7a4-77db398616ad admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:27:50.339932 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29842889-a482-4ef8-807c-45b2ba53c1c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:27:50.341122 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29842889-a482-4ef8-807c-45b2ba53c1c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:27:50.343906 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29842889-a482-4ef8-807c-45b2ba53c1c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:27:50.343906 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29842889-a482-4ef8-807c-45b2ba53c1c3 admin admin] PATH_INFO: `/v3/projects/ad74b940904840ea9be96637bdcd8660/users/f3f6322ca393415f9cc387c822244ca9/roles/15f634d4a2d247ddab90c458e5e1c141` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:27:50.345440 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca00e0e-3903-48db-b261-8734a9305506 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:27:50.350381 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99acfbc3-7134-4401-82d4-f8eb8b086dc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:27:50.351248 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99acfbc3-7134-4401-82d4-f8eb8b086dc4 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:27:50.351755 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99acfbc3-7134-4401-82d4-f8eb8b086dc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:27:50.352313 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99acfbc3-7134-4401-82d4-f8eb8b086dc4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:27:50.353158 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99acfbc3-7134-4401-82d4-f8eb8b086dc4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:27:50.355350 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99acfbc3-7134-4401-82d4-f8eb8b086dc4 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:27:50.362399 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29842889-a482-4ef8-807c-45b2ba53c1c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad74b940904840ea9be96637bdcd8660, user_id=f3f6322ca393415f9cc387c822244ca9, role_id=15f634d4a2d247ddab90c458e5e1c141)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:27:50.365544 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29842889-a482-4ef8-807c-45b2ba53c1c3 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:27:50.367982 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 818/1629] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 24 09:27:50 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:27:50.368934 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99acfbc3-7134-4401-82d4-f8eb8b086dc4 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:27:50.432186 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99acfbc3-7134-4401-82d4-f8eb8b086dc4 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:27:50.436709 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1633215-f5eb-40c4-9169-2ae61170ca6c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c32b83577c46488906fe409a790b24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:27:50.437344 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1633215-f5eb-40c4-9169-2ae61170ca6c service cinder] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:27:50.437622 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1633215-f5eb-40c4-9169-2ae61170ca6c service cinder] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:27:50.437895 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1633215-f5eb-40c4-9169-2ae61170ca6c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:27:50.449186 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1633215-f5eb-40c4-9169-2ae61170ca6c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:27:50.450604 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1633215-f5eb-40c4-9169-2ae61170ca6c service cinder] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:27:50.463486 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 819/1630] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:27:50 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:27:50.464393 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1633215-f5eb-40c4-9169-2ae61170ca6c service cinder] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:27:50.478595 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1633215-f5eb-40c4-9169-2ae61170ca6c service cinder] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:27:50.491954 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 812/1631] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 24 09:27:50 2022] PUT /identity/v3/projects/ad74b940904840ea9be96637bdcd8660/users/f3f6322ca393415f9cc387c822244ca9/roles/15f634d4a2d247ddab90c458e5e1c141 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:27:50.493901 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f609d9b-f29b-42f2-b859-ff007d9dc9a4 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:27:50.494582 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f609d9b-f29b-42f2-b859-ff007d9dc9a4 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:27:50.495060 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f609d9b-f29b-42f2-b859-ff007d9dc9a4 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:27:50.503813 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 813/1632] 10.222.0.16 () {54 vars in 912 bytes} [Wed Aug 24 09:27:50 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:27:50.505349 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f609d9b-f29b-42f2-b859-ff007d9dc9a4 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:27:50.521032 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:27:50.551694 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379f9a5e-a49f-4f79-ba2e-23d2111dd9a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:27:50.553132 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379f9a5e-a49f-4f79-ba2e-23d2111dd9a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:27:50.553481 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379f9a5e-a49f-4f79-ba2e-23d2111dd9a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:27:50.553830 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379f9a5e-a49f-4f79-ba2e-23d2111dd9a3 admin admin] PATH_INFO: `/v3/projects/c23aab1ccdd9406da39a2d3302a81be3/users/9599efe61953407b827ce2a621426633/roles/15f634d4a2d247ddab90c458e5e1c141` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:27:50.576410 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379f9a5e-a49f-4f79-ba2e-23d2111dd9a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c23aab1ccdd9406da39a2d3302a81be3, user_id=9599efe61953407b827ce2a621426633, role_id=15f634d4a2d247ddab90c458e5e1c141)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:27:50.593109 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379f9a5e-a49f-4f79-ba2e-23d2111dd9a3 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:27:50.607173 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f609d9b-f29b-42f2-b859-ff007d9dc9a4 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:27:50.624118 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6034682c-ad07-46f2-a71d-cdaf01caa5e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:27:50.628050 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6034682c-ad07-46f2-a71d-cdaf01caa5e6 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:27:50.628050 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6034682c-ad07-46f2-a71d-cdaf01caa5e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:27:50.628050 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6034682c-ad07-46f2-a71d-cdaf01caa5e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:27:50.628050 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6034682c-ad07-46f2-a71d-cdaf01caa5e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:27:50.629568 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6034682c-ad07-46f2-a71d-cdaf01caa5e6 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:27:50.643560 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 814/1633] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 24 09:27:50 2022] GET /identity/v3/roles => generated 1420 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:27:50.645964 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21242a9b-7b0d-46b2-adff-ab2ad3e5eda8 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:27:50.646345 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21242a9b-7b0d-46b2-adff-ab2ad3e5eda8 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:27:50.647072 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21242a9b-7b0d-46b2-adff-ab2ad3e5eda8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:27:50.675183 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-21242a9b-7b0d-46b2-adff-ab2ad3e5eda8 None None] MFA Rules not processed for user `1f055f7dc51d43bda9b315ca67f11d58`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:27:50.677090 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-21242a9b-7b0d-46b2-adff-ab2ad3e5eda8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:27:50.681027 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 820/1634] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 24 09:27:50 2022] PUT /identity/v3/projects/c23aab1ccdd9406da39a2d3302a81be3/users/9599efe61953407b827ce2a621426633/roles/15f634d4a2d247ddab90c458e5e1c141 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:27:50.686258 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379f9a5e-a49f-4f79-ba2e-23d2111dd9a3 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:27:50.701942 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:27:50.810138 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379f9a5e-a49f-4f79-ba2e-23d2111dd9a3 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:27:50.815899 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe2d84d-96ea-46a0-a7e0-4053c06caf86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:27:50.816726 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe2d84d-96ea-46a0-a7e0-4053c06caf86 admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:27:50.817079 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe2d84d-96ea-46a0-a7e0-4053c06caf86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:27:50.817365 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe2d84d-96ea-46a0-a7e0-4053c06caf86 admin admin] PATH_INFO: `/v3/projects/ad74b940904840ea9be96637bdcd8660/users/f3f6322ca393415f9cc387c822244ca9/roles/15f634d4a2d247ddab90c458e5e1c141` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:27:50.819498 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe2d84d-96ea-46a0-a7e0-4053c06caf86 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad74b940904840ea9be96637bdcd8660, user_id=f3f6322ca393415f9cc387c822244ca9, role_id=15f634d4a2d247ddab90c458e5e1c141)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:27:50.823541 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe2d84d-96ea-46a0-a7e0-4053c06caf86 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:27:50.854564 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 815/1636] 10.222.0.16 () {58 vars in 1009 bytes} [Wed Aug 24 09:27:50 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:27:50.854564 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21242a9b-7b0d-46b2-adff-ab2ad3e5eda8 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:27:50.855911 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 821/1636] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 24 09:27:50 2022] PUT /identity/v3/projects/ad74b940904840ea9be96637bdcd8660/users/f3f6322ca393415f9cc387c822244ca9/roles/15f634d4a2d247ddab90c458e5e1c141 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:27:50.864899 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21242a9b-7b0d-46b2-adff-ab2ad3e5eda8 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:27:50.865806 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe2d84d-96ea-46a0-a7e0-4053c06caf86 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:27:50.987113 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc278ef5-8a3d-45d4-a18b-0ac29f5cb472 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:27:50.987982 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc278ef5-8a3d-45d4-a18b-0ac29f5cb472 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:27:50.988272 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc278ef5-8a3d-45d4-a18b-0ac29f5cb472 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:27:50.988550 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc278ef5-8a3d-45d4-a18b-0ac29f5cb472 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:27:50.989208 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc278ef5-8a3d-45d4-a18b-0ac29f5cb472 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:27:50.991211 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc278ef5-8a3d-45d4-a18b-0ac29f5cb472 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:27:50.997823 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe2d84d-96ea-46a0-a7e0-4053c06caf86 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:27:51.013509 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 816/1637] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 24 09:27:50 2022] GET /identity/v3/roles => generated 1420 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:27:51.015507 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ae2d17-daa2-45bd-bed4-8cd0acd72c04 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:27:51.015649 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ae2d17-daa2-45bd-bed4-8cd0acd72c04 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:27:51.015884 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ae2d17-daa2-45bd-bed4-8cd0acd72c04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:27:51.019110 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37feb8b-c7b2-4e22-9979-a8c162859fcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:27:51.020299 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37feb8b-c7b2-4e22-9979-a8c162859fcf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:27:51.020813 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37feb8b-c7b2-4e22-9979-a8c162859fcf admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:27:51.021185 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37feb8b-c7b2-4e22-9979-a8c162859fcf admin admin] PATH_INFO: `/v3/users/b3d64b436fc8454087a68c0506264de6` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:27:51.022499 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37feb8b-c7b2-4e22-9979-a8c162859fcf admin admin] RBAC: Authorizing `identity:delete_user(user_id=b3d64b436fc8454087a68c0506264de6)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:27:51.024741 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37feb8b-c7b2-4e22-9979-a8c162859fcf admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:27:51.049867 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-74ae2d17-daa2-45bd-bed4-8cd0acd72c04 None None] MFA Rules not processed for user `f3f6322ca393415f9cc387c822244ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:27:51.051116 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-74ae2d17-daa2-45bd-bed4-8cd0acd72c04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:27:51.152669 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 817/1638] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 09:27:51 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:27:51.153846 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ae2d17-daa2-45bd-bed4-8cd0acd72c04 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:27:51.193257 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:27:51.237399 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-f37feb8b-c7b2-4e22-9979-a8c162859fcf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b3d64b436fc8454087a68c0506264de6'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:27:51.238695 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-f37feb8b-c7b2-4e22-9979-a8c162859fcf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b3d64b436fc8454087a68c0506264de6'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:27:51.245801 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ae2d17-daa2-45bd-bed4-8cd0acd72c04 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:27:51.248100 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:27:51.252605 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7db5349-b34f-43fb-911f-cb3c825fd62d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c32b83577c46488906fe409a790b24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:27:51.253378 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7db5349-b34f-43fb-911f-cb3c825fd62d service cinder] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:27:51.253664 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7db5349-b34f-43fb-911f-cb3c825fd62d service cinder] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:27:51.255699 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7db5349-b34f-43fb-911f-cb3c825fd62d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:27:51.474766 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-f37feb8b-c7b2-4e22-9979-a8c162859fcf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b3d64b436fc8454087a68c0506264de6'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:27:51.474766 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-f37feb8b-c7b2-4e22-9979-a8c162859fcf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b3d64b436fc8454087a68c0506264de6'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:27:51.474766 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-f37feb8b-c7b2-4e22-9979-a8c162859fcf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b3d64b436fc8454087a68c0506264de6'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:27:51.474766 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 822/1639] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 24 09:27:50 2022] DELETE /identity/v3/users/b3d64b436fc8454087a68c0506264de6 => generated 0 bytes in 516 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:27:51.474766 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37feb8b-c7b2-4e22-9979-a8c162859fcf admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:27:51.475494 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-f37feb8b-c7b2-4e22-9979-a8c162859fcf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:27:51.476135 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7db5349-b34f-43fb-911f-cb3c825fd62d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:27:51.476135 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7db5349-b34f-43fb-911f-cb3c825fd62d service cinder] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:27:51.476135 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 818/1640] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:27:51 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:27:51.476135 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7db5349-b34f-43fb-911f-cb3c825fd62d service cinder] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:27:51.476135 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7db5349-b34f-43fb-911f-cb3c825fd62d service cinder] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:27:51.523488 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37feb8b-c7b2-4e22-9979-a8c162859fcf admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:27:51.523488 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b58b9d4-99e4-401e-ae4f-f75dcc562a84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:27:51.523488 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b58b9d4-99e4-401e-ae4f-f75dcc562a84 admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:27:51.523488 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b58b9d4-99e4-401e-ae4f-f75dcc562a84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:27:51.524312 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b58b9d4-99e4-401e-ae4f-f75dcc562a84 admin admin] PATH_INFO: `/v3/projects/c23aab1ccdd9406da39a2d3302a81be3/users/9599efe61953407b827ce2a621426633/roles/15f634d4a2d247ddab90c458e5e1c141` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:27:51.524312 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b58b9d4-99e4-401e-ae4f-f75dcc562a84 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c23aab1ccdd9406da39a2d3302a81be3, user_id=9599efe61953407b827ce2a621426633, role_id=15f634d4a2d247ddab90c458e5e1c141)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:27:51.524312 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b58b9d4-99e4-401e-ae4f-f75dcc562a84 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:27:51.524502 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7552d36f-c692-4755-b464-fa6d23e07ffa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c32b83577c46488906fe409a790b24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:27:51.524502 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7552d36f-c692-4755-b464-fa6d23e07ffa service cinder] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:27:51.524502 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7552d36f-c692-4755-b464-fa6d23e07ffa service cinder] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:27:51.524502 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7552d36f-c692-4755-b464-fa6d23e07ffa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:27:51.526076 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-7552d36f-c692-4755-b464-fa6d23e07ffa service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:27:51.537432 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 823/1641] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 24 09:27:51 2022] PUT /identity/v3/projects/c23aab1ccdd9406da39a2d3302a81be3/users/9599efe61953407b827ce2a621426633/roles/15f634d4a2d247ddab90c458e5e1c141 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:27:51.539678 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e59ff15-f8ea-448a-b483-914d7a8e2a8a None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:27:51.539973 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e59ff15-f8ea-448a-b483-914d7a8e2a8a None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:27:51.540253 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e59ff15-f8ea-448a-b483-914d7a8e2a8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:27:51.604647 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-5e59ff15-f8ea-448a-b483-914d7a8e2a8a None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:27:51.607204 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-5e59ff15-f8ea-448a-b483-914d7a8e2a8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:27:51.663037 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7552d36f-c692-4755-b464-fa6d23e07ffa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:27:51.666975 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7552d36f-c692-4755-b464-fa6d23e07ffa service cinder] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:27:51.678111 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 819/1642] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:27:51 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 276 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:27:51.679529 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4c318c-c420-444c-832e-abecc9d131b0 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:27:51.679842 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4c318c-c420-444c-832e-abecc9d131b0 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:27:51.680116 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4c318c-c420-444c-832e-abecc9d131b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:27:51.714325 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-ca4c318c-c420-444c-832e-abecc9d131b0 None None] MFA Rules not processed for user `9599efe61953407b827ce2a621426633`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:27:51.715849 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-ca4c318c-c420-444c-832e-abecc9d131b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:27:51.726848 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:27:51.779759 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 824/1643] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 09:27:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:27:51.790071 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e59ff15-f8ea-448a-b483-914d7a8e2a8a None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:27:51.791843 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-5e59ff15-f8ea-448a-b483-914d7a8e2a8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:27:51.810316 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 820/1644] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 09:27:51 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:27:51.864659 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e59ff15-f8ea-448a-b483-914d7a8e2a8a None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:27:51.869610 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc4eb59-f2f8-4148-a521-3211eadf8077 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e5260e798404e8bb12b25ca8ea97052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:27:51.870668 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc4eb59-f2f8-4148-a521-3211eadf8077 service neutron] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:27:51.871028 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc4eb59-f2f8-4148-a521-3211eadf8077 service neutron] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:27:51.871374 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc4eb59-f2f8-4148-a521-3211eadf8077 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:27:51.915534 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc4eb59-f2f8-4148-a521-3211eadf8077 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:27:51.917049 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc4eb59-f2f8-4148-a521-3211eadf8077 service neutron] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:27:51.937537 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 825/1645] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:27:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 09:27:52.673404 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4c318c-c420-444c-832e-abecc9d131b0 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:27:52.674922 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-ca4c318c-c420-444c-832e-abecc9d131b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:27:52.775538 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4c318c-c420-444c-832e-abecc9d131b0 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:27:52.785241 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a49b041-a5a3-4968-aa7d-fad1fce835e0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c32b83577c46488906fe409a790b24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:27:52.785905 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a49b041-a5a3-4968-aa7d-fad1fce835e0 service cinder] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:27:52.786172 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a49b041-a5a3-4968-aa7d-fad1fce835e0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:27:52.786527 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a49b041-a5a3-4968-aa7d-fad1fce835e0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:27:52.799109 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a49b041-a5a3-4968-aa7d-fad1fce835e0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:27:52.800837 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a49b041-a5a3-4968-aa7d-fad1fce835e0 service cinder] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:27:52.845329 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 821/1646] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:27:52 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:27:53.613367 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc4eb59-f2f8-4148-a521-3211eadf8077 service neutron] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:27:53.614859 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-9dc4eb59-f2f8-4148-a521-3211eadf8077 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:27:53.640584 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc4eb59-f2f8-4148-a521-3211eadf8077 service neutron] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:27:53.652248 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebf50c6-cf0b-4883-a632-ceebb6c7b48d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:27:53.653374 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebf50c6-cf0b-4883-a632-ceebb6c7b48d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:27:53.653938 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebf50c6-cf0b-4883-a632-ceebb6c7b48d admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:27:53.655313 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebf50c6-cf0b-4883-a632-ceebb6c7b48d admin admin] PATH_INFO: `/v3/projects/0d04a503a13a48439073f044775b24a1` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:27:53.656814 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebf50c6-cf0b-4883-a632-ceebb6c7b48d admin admin] RBAC: Authorizing `identity:delete_project(project_id=0d04a503a13a48439073f044775b24a1)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:27:53.659116 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: WARNING py.warnings [None req-2ebf50c6-cf0b-4883-a632-ceebb6c7b48d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 09:27:53.659116 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: warnings.warn(msg) Aug 24 09:27:53.659116 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: Aug 24 09:27:53.662332 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebf50c6-cf0b-4883-a632-ceebb6c7b48d admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:27:53.666462 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=96,keystone:DELETE=2,keystone:INSERT=2 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:27:53.944833 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-2ebf50c6-cf0b-4883-a632-ceebb6c7b48d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0d04a503a13a48439073f044775b24a1', 'request_id': 'req-2ebf50c6-cf0b-4883-a632-ceebb6c7b48d'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:27:53.970144 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-2ebf50c6-cf0b-4883-a632-ceebb6c7b48d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0d04a503a13a48439073f044775b24a1', 'request_id': 'req-2ebf50c6-cf0b-4883-a632-ceebb6c7b48d'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:27:53.990423 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-2ebf50c6-cf0b-4883-a632-ceebb6c7b48d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0d04a503a13a48439073f044775b24a1', 'request_id': 'req-2ebf50c6-cf0b-4883-a632-ceebb6c7b48d'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:27:54.183321 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-2ebf50c6-cf0b-4883-a632-ceebb6c7b48d admin admin] The token cache is being invalidate because project 0d04a503a13a48439073f044775b24a1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:27:54.183648 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-2ebf50c6-cf0b-4883-a632-ceebb6c7b48d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:27:54.184354 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-2ebf50c6-cf0b-4883-a632-ceebb6c7b48d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:27:54.186718 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 826/1647] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 24 09:27:53 2022] DELETE /identity/v3/projects/0d04a503a13a48439073f044775b24a1 => generated 0 bytes in 574 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:27:54.191243 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8afda0-ed4c-4c4a-90b0-46e89510760d None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:27:54.191800 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8afda0-ed4c-4c4a-90b0-46e89510760d None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:27:54.191800 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8afda0-ed4c-4c4a-90b0-46e89510760d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:27:54.228874 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-ba8afda0-ed4c-4c4a-90b0-46e89510760d None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:27:54.230137 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-ba8afda0-ed4c-4c4a-90b0-46e89510760d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:27:54.391665 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 822/1648] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 09:27:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:27:54.396858 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfd99c5-efaf-4af4-b4cc-e61bd5c36997 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:27:54.396858 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfd99c5-efaf-4af4-b4cc-e61bd5c36997 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:27:54.396858 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfd99c5-efaf-4af4-b4cc-e61bd5c36997 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:27:54.434418 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-2cfd99c5-efaf-4af4-b4cc-e61bd5c36997 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:27:54.436201 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-2cfd99c5-efaf-4af4-b4cc-e61bd5c36997 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:27:54.458020 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 827/1649] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 09:27:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:27:54.465367 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8afda0-ed4c-4c4a-90b0-46e89510760d None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:27:54.480885 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8afda0-ed4c-4c4a-90b0-46e89510760d None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:27:54.489662 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05628515-4861-4d6e-8018-9745202bf365 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:27:54.490695 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05628515-4861-4d6e-8018-9745202bf365 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:27:54.491120 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05628515-4861-4d6e-8018-9745202bf365 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:27:54.491618 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05628515-4861-4d6e-8018-9745202bf365 admin admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:27:54.493211 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05628515-4861-4d6e-8018-9745202bf365 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:27:54.493733 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05628515-4861-4d6e-8018-9745202bf365 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:27:54.498103 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05628515-4861-4d6e-8018-9745202bf365 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:27:54.557121 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 823/1650] 10.222.0.16 () {58 vars in 1153 bytes} [Wed Aug 24 09:27:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:27:54.557816 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfd99c5-efaf-4af4-b4cc-e61bd5c36997 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:27:54.572803 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfd99c5-efaf-4af4-b4cc-e61bd5c36997 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:27:54.581616 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8584b02d-2f98-4818-a8cb-42ceb2a349a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:27:54.585440 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8584b02d-2f98-4818-a8cb-42ceb2a349a1 admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:27:54.586040 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8584b02d-2f98-4818-a8cb-42ceb2a349a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:27:54.586515 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8584b02d-2f98-4818-a8cb-42ceb2a349a1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:27:54.588002 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8584b02d-2f98-4818-a8cb-42ceb2a349a1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:27:54.598733 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8584b02d-2f98-4818-a8cb-42ceb2a349a1 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:27:54.765055 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05628515-4861-4d6e-8018-9745202bf365 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:27:54.772743 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-05628515-4861-4d6e-8018-9745202bf365 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:27:54.788711 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 828/1651] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 24 09:27:54 2022] POST /identity/v3/projects => generated 369 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 09:27:54.793631 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8584b02d-2f98-4818-a8cb-42ceb2a349a1 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:27:54.815865 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05628515-4861-4d6e-8018-9745202bf365 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:27:54.918853 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:27:54.978033 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8584b02d-2f98-4818-a8cb-42ceb2a349a1 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:27:54.982093 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28941e7a-6ecd-4145-9120-090df5ada2b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:27:54.983873 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28941e7a-6ecd-4145-9120-090df5ada2b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:27:54.984708 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28941e7a-6ecd-4145-9120-090df5ada2b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:27:54.985104 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28941e7a-6ecd-4145-9120-090df5ada2b8 admin admin] PATH_INFO: `/v3/users/862990dac5f84b07abc9fe26ccd6a41a` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:27:54.990527 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28941e7a-6ecd-4145-9120-090df5ada2b8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=862990dac5f84b07abc9fe26ccd6a41a)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:27:54.993388 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: WARNING py.warnings [None req-28941e7a-6ecd-4145-9120-090df5ada2b8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 09:27:54.993388 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: warnings.warn(msg) Aug 24 09:27:54.993388 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: Aug 24 09:27:54.995092 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28941e7a-6ecd-4145-9120-090df5ada2b8 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:27:54.999439 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464a2d08-ca21-4053-8d9e-4e50eb5b5999 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:27:54.999538 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464a2d08-ca21-4053-8d9e-4e50eb5b5999 admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:27:55.000028 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464a2d08-ca21-4053-8d9e-4e50eb5b5999 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:27:55.000445 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464a2d08-ca21-4053-8d9e-4e50eb5b5999 admin admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:27:55.001504 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464a2d08-ca21-4053-8d9e-4e50eb5b5999 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:27:55.021506 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464a2d08-ca21-4053-8d9e-4e50eb5b5999 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:27:55.245398 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 829/1652] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 24 09:27:54 2022] POST /identity/v3/users => generated 285 bytes in 456 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:27:55.253331 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464a2d08-ca21-4053-8d9e-4e50eb5b5999 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:27:55.292490 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464a2d08-ca21-4053-8d9e-4e50eb5b5999 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:27:55.315286 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b426a36b-643c-4846-8528-c0b55139b98a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:27:55.316139 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b426a36b-643c-4846-8528-c0b55139b98a admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:27:55.316439 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b426a36b-643c-4846-8528-c0b55139b98a admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:27:55.316780 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b426a36b-643c-4846-8528-c0b55139b98a admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:27:55.317468 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b426a36b-643c-4846-8528-c0b55139b98a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:27:55.330916 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b426a36b-643c-4846-8528-c0b55139b98a admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:27:55.382960 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 830/1653] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 24 09:27:55 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:27:55.387929 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b426a36b-643c-4846-8528-c0b55139b98a admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:27:55.411516 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-28941e7a-6ecd-4145-9120-090df5ada2b8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '862990dac5f84b07abc9fe26ccd6a41a'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:27:55.416649 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-28941e7a-6ecd-4145-9120-090df5ada2b8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '862990dac5f84b07abc9fe26ccd6a41a'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:27:55.423100 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b426a36b-643c-4846-8528-c0b55139b98a admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:27:55.451146 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d4cfdd-6244-4d5c-80b0-b1f58eef4d42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:27:55.451146 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d4cfdd-6244-4d5c-80b0-b1f58eef4d42 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:27:55.451146 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d4cfdd-6244-4d5c-80b0-b1f58eef4d42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:27:55.451146 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d4cfdd-6244-4d5c-80b0-b1f58eef4d42 admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:27:55.452067 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d4cfdd-6244-4d5c-80b0-b1f58eef4d42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:27:55.453126 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d4cfdd-6244-4d5c-80b0-b1f58eef4d42 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:27:55.480640 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 831/1654] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 24 09:27:55 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:27:55.485067 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d4cfdd-6244-4d5c-80b0-b1f58eef4d42 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:27:55.491002 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-28941e7a-6ecd-4145-9120-090df5ada2b8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '862990dac5f84b07abc9fe26ccd6a41a'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:27:55.492181 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-28941e7a-6ecd-4145-9120-090df5ada2b8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '862990dac5f84b07abc9fe26ccd6a41a'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:27:55.509869 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d4cfdd-6244-4d5c-80b0-b1f58eef4d42 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:27:55.515670 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-28941e7a-6ecd-4145-9120-090df5ada2b8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '862990dac5f84b07abc9fe26ccd6a41a'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:27:55.521710 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcbfecb-6220-4097-8904-8ac24fac3d3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:27:55.522506 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcbfecb-6220-4097-8904-8ac24fac3d3f admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:27:55.523427 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcbfecb-6220-4097-8904-8ac24fac3d3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:27:55.523962 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcbfecb-6220-4097-8904-8ac24fac3d3f admin admin] PATH_INFO: `/v3/projects/6a5c0a6059034b7b880f1b3768064ce0/users/e93fc592784e4dd7b22e8fa596c6cee5/roles/136cdf78e00a426997dd4e37faff489b` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:27:55.573854 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcbfecb-6220-4097-8904-8ac24fac3d3f admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a5c0a6059034b7b880f1b3768064ce0, user_id=e93fc592784e4dd7b22e8fa596c6cee5, role_id=136cdf78e00a426997dd4e37faff489b)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:27:55.578387 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcbfecb-6220-4097-8904-8ac24fac3d3f admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:27:55.650699 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 824/1655] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 24 09:27:54 2022] DELETE /identity/v3/users/862990dac5f84b07abc9fe26ccd6a41a => generated 0 bytes in 879 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:27:55.652404 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bda95f4-66ca-4104-baa9-44cebf59994e None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:27:55.652404 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bda95f4-66ca-4104-baa9-44cebf59994e None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:27:55.655030 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bda95f4-66ca-4104-baa9-44cebf59994e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:27:55.671299 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 832/1656] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 24 09:27:55 2022] PUT /identity/v3/projects/6a5c0a6059034b7b880f1b3768064ce0/users/e93fc592784e4dd7b22e8fa596c6cee5/roles/136cdf78e00a426997dd4e37faff489b => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:27:55.681018 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df63d207-62a5-4ea4-92f5-b54e38bb3733 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:27:55.683956 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df63d207-62a5-4ea4-92f5-b54e38bb3733 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:27:55.684243 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df63d207-62a5-4ea4-92f5-b54e38bb3733 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:27:55.738785 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=87,keystone:DELETE=7,keystone:INSERT=6 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:27:55.756804 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-6bda95f4-66ca-4104-baa9-44cebf59994e None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:27:55.760296 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-6bda95f4-66ca-4104-baa9-44cebf59994e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:27:55.801230 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-df63d207-62a5-4ea4-92f5-b54e38bb3733 None None] MFA Rules not processed for user `e93fc592784e4dd7b22e8fa596c6cee5`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:27:55.803418 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-df63d207-62a5-4ea4-92f5-b54e38bb3733 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:27:55.924245 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 833/1657] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 09:27:55 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:27:55.933401 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df63d207-62a5-4ea4-92f5-b54e38bb3733 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:27:55.936885 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-df63d207-62a5-4ea4-92f5-b54e38bb3733 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:27:55.973138 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 825/1658] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 09:27:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:27:55.994034 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df63d207-62a5-4ea4-92f5-b54e38bb3733 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:27:56.002609 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9de2772-52b3-4099-89f4-af403441ceac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:27:56.003326 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9de2772-52b3-4099-89f4-af403441ceac admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:27:56.003698 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9de2772-52b3-4099-89f4-af403441ceac admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:27:56.003979 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9de2772-52b3-4099-89f4-af403441ceac admin admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:27:56.004767 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9de2772-52b3-4099-89f4-af403441ceac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:27:56.006691 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9de2772-52b3-4099-89f4-af403441ceac admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:27:56.019198 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bda95f4-66ca-4104-baa9-44cebf59994e None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:27:56.021131 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-6bda95f4-66ca-4104-baa9-44cebf59994e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:27:56.157540 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bda95f4-66ca-4104-baa9-44cebf59994e None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:27:56.168435 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 834/1659] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 24 09:27:55 2022] POST /identity/v3/projects => generated 367 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:27:56.168760 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249c31ae-83f4-4592-992c-a8a0d5e0f7c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e5260e798404e8bb12b25ca8ea97052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:27:56.168760 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249c31ae-83f4-4592-992c-a8a0d5e0f7c8 service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:27:56.168760 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249c31ae-83f4-4592-992c-a8a0d5e0f7c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:27:56.169095 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249c31ae-83f4-4592-992c-a8a0d5e0f7c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:27:56.170158 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9de2772-52b3-4099-89f4-af403441ceac admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:27:56.193448 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249c31ae-83f4-4592-992c-a8a0d5e0f7c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:27:56.194976 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249c31ae-83f4-4592-992c-a8a0d5e0f7c8 service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:27:56.199562 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:27:56.211220 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 826/1660] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:27:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:27:56.353352 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9de2772-52b3-4099-89f4-af403441ceac admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:27:56.370713 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d10f430-8ec5-40bb-a8e8-8d5befacc1bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:27:56.371644 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d10f430-8ec5-40bb-a8e8-8d5befacc1bc admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:27:56.372069 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d10f430-8ec5-40bb-a8e8-8d5befacc1bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:27:56.372479 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d10f430-8ec5-40bb-a8e8-8d5befacc1bc admin admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:27:56.377319 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d10f430-8ec5-40bb-a8e8-8d5befacc1bc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:27:56.381866 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d10f430-8ec5-40bb-a8e8-8d5befacc1bc admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:27:56.526530 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 835/1661] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 24 09:27:56 2022] POST /identity/v3/users => generated 290 bytes in 357 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:27:56.531900 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249c31ae-83f4-4592-992c-a8a0d5e0f7c8 service neutron] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:27:56.548227 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249c31ae-83f4-4592-992c-a8a0d5e0f7c8 service neutron] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:27:56.565347 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1206403b-881a-482c-beb9-b5021330e128 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:27:56.566409 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1206403b-881a-482c-beb9-b5021330e128 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:27:56.566776 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1206403b-881a-482c-beb9-b5021330e128 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:27:56.567059 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1206403b-881a-482c-beb9-b5021330e128 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:27:56.567782 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1206403b-881a-482c-beb9-b5021330e128 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:27:56.573358 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1206403b-881a-482c-beb9-b5021330e128 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:27:56.601805 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 827/1662] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 24 09:27:56 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:27:56.606297 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d10f430-8ec5-40bb-a8e8-8d5befacc1bc admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:27:56.625057 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d10f430-8ec5-40bb-a8e8-8d5befacc1bc admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:27:56.630238 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832ef19f-81fd-4056-a19c-a3634268d4d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:27:56.631009 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832ef19f-81fd-4056-a19c-a3634268d4d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:27:56.631296 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832ef19f-81fd-4056-a19c-a3634268d4d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:27:56.631574 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832ef19f-81fd-4056-a19c-a3634268d4d1 admin admin] PATH_INFO: `/v3/projects/33e5b4a408f1463a848113782742f8cd/users/1a142d6b2a774637a141aa10d15802c8/roles/15f634d4a2d247ddab90c458e5e1c141` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:27:56.655870 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832ef19f-81fd-4056-a19c-a3634268d4d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=33e5b4a408f1463a848113782742f8cd, user_id=1a142d6b2a774637a141aa10d15802c8, role_id=15f634d4a2d247ddab90c458e5e1c141)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:27:56.664074 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832ef19f-81fd-4056-a19c-a3634268d4d1 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:27:56.789217 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 836/1663] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 24 09:27:56 2022] PUT /identity/v3/projects/33e5b4a408f1463a848113782742f8cd/users/1a142d6b2a774637a141aa10d15802c8/roles/15f634d4a2d247ddab90c458e5e1c141 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:27:56.793209 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1206403b-881a-482c-beb9-b5021330e128 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:27:56.826515 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1206403b-881a-482c-beb9-b5021330e128 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:27:57.037019 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53125240-0675-443b-8114-1923cbf416ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:27:57.037722 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53125240-0675-443b-8114-1923cbf416ae admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:27:57.038005 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53125240-0675-443b-8114-1923cbf416ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:27:57.040833 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53125240-0675-443b-8114-1923cbf416ae admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:27:57.041775 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53125240-0675-443b-8114-1923cbf416ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:27:57.047560 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53125240-0675-443b-8114-1923cbf416ae admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:27:57.085760 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 828/1664] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 24 09:27:56 2022] GET /identity/v3/roles => generated 1420 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:27:57.088215 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832ef19f-81fd-4056-a19c-a3634268d4d1 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:27:57.125852 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832ef19f-81fd-4056-a19c-a3634268d4d1 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:27:57.148545 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2199e93-d106-4701-8af0-30799aaaf589 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:27:57.151947 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2199e93-d106-4701-8af0-30799aaaf589 admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:27:57.153762 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2199e93-d106-4701-8af0-30799aaaf589 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:27:57.155030 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2199e93-d106-4701-8af0-30799aaaf589 admin admin] PATH_INFO: `/v3/projects/33e5b4a408f1463a848113782742f8cd/users/1a142d6b2a774637a141aa10d15802c8/roles/15f634d4a2d247ddab90c458e5e1c141` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:27:57.163868 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2199e93-d106-4701-8af0-30799aaaf589 admin admin] RBAC: Authorizing `identity:create_grant(project_id=33e5b4a408f1463a848113782742f8cd, user_id=1a142d6b2a774637a141aa10d15802c8, role_id=15f634d4a2d247ddab90c458e5e1c141)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:27:57.178398 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53125240-0675-443b-8114-1923cbf416ae admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:27:57.185079 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-53125240-0675-443b-8114-1923cbf416ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:27:57.195723 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2199e93-d106-4701-8af0-30799aaaf589 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:27:57.240416 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53125240-0675-443b-8114-1923cbf416ae admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:27:57.265286 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 837/1665] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 24 09:27:57 2022] PUT /identity/v3/projects/33e5b4a408f1463a848113782742f8cd/users/1a142d6b2a774637a141aa10d15802c8/roles/15f634d4a2d247ddab90c458e5e1c141 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:27:57.277227 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9d6886-b59f-4b18-9341-30b872b1052e None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:27:57.277521 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9d6886-b59f-4b18-9341-30b872b1052e None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:27:57.277794 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9d6886-b59f-4b18-9341-30b872b1052e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:27:57.365307 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-8b9d6886-b59f-4b18-9341-30b872b1052e None None] MFA Rules not processed for user `1a142d6b2a774637a141aa10d15802c8`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:27:57.366709 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-8b9d6886-b59f-4b18-9341-30b872b1052e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:27:57.424877 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:27:57.424975 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd0c8b3-d7c0-4243-9dd4-667cb438dfae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:27:57.424975 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd0c8b3-d7c0-4243-9dd4-667cb438dfae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:27:57.424975 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd0c8b3-d7c0-4243-9dd4-667cb438dfae admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:27:57.424975 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd0c8b3-d7c0-4243-9dd4-667cb438dfae admin admin] PATH_INFO: `/v3/users/a99817614d86407ea10902774e47fca2` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:27:57.425280 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd0c8b3-d7c0-4243-9dd4-667cb438dfae admin admin] RBAC: Authorizing `identity:delete_user(user_id=a99817614d86407ea10902774e47fca2)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:27:57.425280 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd0c8b3-d7c0-4243-9dd4-667cb438dfae admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:27:57.519951 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 838/1666] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 09:27:57 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 09:27:57.531814 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9d6886-b59f-4b18-9341-30b872b1052e None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:27:57.533636 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-8b9d6886-b59f-4b18-9341-30b872b1052e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:27:57.607660 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-7fd0c8b3-d7c0-4243-9dd4-667cb438dfae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a99817614d86407ea10902774e47fca2'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:27:57.608968 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-7fd0c8b3-d7c0-4243-9dd4-667cb438dfae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a99817614d86407ea10902774e47fca2'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:27:57.627876 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9d6886-b59f-4b18-9341-30b872b1052e None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:27:57.637628 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36db4f5b-99c3-4d14-81a4-11d2b81726dc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c32b83577c46488906fe409a790b24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:27:57.637628 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36db4f5b-99c3-4d14-81a4-11d2b81726dc service cinder] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:27:57.637628 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36db4f5b-99c3-4d14-81a4-11d2b81726dc service cinder] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:27:57.637628 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36db4f5b-99c3-4d14-81a4-11d2b81726dc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:27:57.650108 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36db4f5b-99c3-4d14-81a4-11d2b81726dc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:27:57.652217 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36db4f5b-99c3-4d14-81a4-11d2b81726dc service cinder] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:27:57.673029 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 839/1667] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:27:57 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 09:27:57.685388 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-7fd0c8b3-d7c0-4243-9dd4-667cb438dfae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a99817614d86407ea10902774e47fca2'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:27:57.687155 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-7fd0c8b3-d7c0-4243-9dd4-667cb438dfae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a99817614d86407ea10902774e47fca2'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:27:57.696966 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-7fd0c8b3-d7c0-4243-9dd4-667cb438dfae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a99817614d86407ea10902774e47fca2'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:27:57.721841 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 829/1668] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 24 09:27:57 2022] DELETE /identity/v3/users/a99817614d86407ea10902774e47fca2 => generated 0 bytes in 541 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 09:27:57.723907 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc48c4ba-f81b-483b-8c78-11bd242fae9a None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:27:57.724092 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc48c4ba-f81b-483b-8c78-11bd242fae9a None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:27:57.724434 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc48c4ba-f81b-483b-8c78-11bd242fae9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:27:57.758564 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-bc48c4ba-f81b-483b-8c78-11bd242fae9a None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:27:57.758564 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-bc48c4ba-f81b-483b-8c78-11bd242fae9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:27:57.947285 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 840/1669] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 09:27:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:27:57.977007 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd0c8b3-d7c0-4243-9dd4-667cb438dfae admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:27:57.979427 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-7fd0c8b3-d7c0-4243-9dd4-667cb438dfae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:27:57.985394 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:27:58.098226 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc48c4ba-f81b-483b-8c78-11bd242fae9a None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:27:58.102174 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-bc48c4ba-f81b-483b-8c78-11bd242fae9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:27:58.131664 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc48c4ba-f81b-483b-8c78-11bd242fae9a None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:27:58.148957 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c47d11-be96-4c1f-83eb-7e3b424ccaed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:27:58.149628 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c47d11-be96-4c1f-83eb-7e3b424ccaed admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:27:58.150221 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c47d11-be96-4c1f-83eb-7e3b424ccaed admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:27:58.150221 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c47d11-be96-4c1f-83eb-7e3b424ccaed admin admin] PATH_INFO: `/v3/projects/05dbe88d418a40bea60335d5411e9d1e` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:27:58.155060 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd0c8b3-d7c0-4243-9dd4-667cb438dfae admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:27:58.157476 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c47d11-be96-4c1f-83eb-7e3b424ccaed admin admin] RBAC: Authorizing `identity:delete_project(project_id=05dbe88d418a40bea60335d5411e9d1e)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:27:58.169241 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c47d11-be96-4c1f-83eb-7e3b424ccaed admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:27:58.170578 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea22b23-0603-4697-9fad-0731112831fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e5260e798404e8bb12b25ca8ea97052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:27:58.171021 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea22b23-0603-4697-9fad-0731112831fd service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:27:58.171309 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea22b23-0603-4697-9fad-0731112831fd service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:27:58.183450 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea22b23-0603-4697-9fad-0731112831fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:27:58.227645 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea22b23-0603-4697-9fad-0731112831fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:27:58.232664 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea22b23-0603-4697-9fad-0731112831fd service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:27:58.264375 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 830/1670] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:27:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:27:58.451004 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-16c47d11-be96-4c1f-83eb-7e3b424ccaed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '05dbe88d418a40bea60335d5411e9d1e', 'request_id': 'req-16c47d11-be96-4c1f-83eb-7e3b424ccaed'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:27:58.463649 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:27:58.484551 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-16c47d11-be96-4c1f-83eb-7e3b424ccaed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '05dbe88d418a40bea60335d5411e9d1e', 'request_id': 'req-16c47d11-be96-4c1f-83eb-7e3b424ccaed'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:27:58.497597 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-16c47d11-be96-4c1f-83eb-7e3b424ccaed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '05dbe88d418a40bea60335d5411e9d1e', 'request_id': 'req-16c47d11-be96-4c1f-83eb-7e3b424ccaed'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:27:58.586709 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-16c47d11-be96-4c1f-83eb-7e3b424ccaed admin admin] The token cache is being invalidate because project 05dbe88d418a40bea60335d5411e9d1e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:27:58.587279 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-16c47d11-be96-4c1f-83eb-7e3b424ccaed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:27:58.588029 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-16c47d11-be96-4c1f-83eb-7e3b424ccaed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:27:58.590472 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 841/1671] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 24 09:27:58 2022] DELETE /identity/v3/projects/05dbe88d418a40bea60335d5411e9d1e => generated 0 bytes in 493 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:27:58.599063 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea22b23-0603-4697-9fad-0731112831fd service neutron] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:27:58.609206 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea22b23-0603-4697-9fad-0731112831fd service neutron] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:27:58.711690 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2e27d2-c94c-481a-8a2b-c37724816c8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:27:58.712823 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2e27d2-c94c-481a-8a2b-c37724816c8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:27:58.713111 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2e27d2-c94c-481a-8a2b-c37724816c8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:27:58.713390 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2e27d2-c94c-481a-8a2b-c37724816c8c admin admin] PATH_INFO: `/v3/users/9599efe61953407b827ce2a621426633` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:27:58.718299 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2e27d2-c94c-481a-8a2b-c37724816c8c admin admin] RBAC: Authorizing `identity:delete_user(user_id=9599efe61953407b827ce2a621426633)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:27:58.721574 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2e27d2-c94c-481a-8a2b-c37724816c8c admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:27:59.056118 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-2d2e27d2-c94c-481a-8a2b-c37724816c8c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9599efe61953407b827ce2a621426633'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:27:59.063512 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-2d2e27d2-c94c-481a-8a2b-c37724816c8c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9599efe61953407b827ce2a621426633'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:27:59.125972 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-2d2e27d2-c94c-481a-8a2b-c37724816c8c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9599efe61953407b827ce2a621426633'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:27:59.126754 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-2d2e27d2-c94c-481a-8a2b-c37724816c8c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9599efe61953407b827ce2a621426633'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:27:59.135658 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-2d2e27d2-c94c-481a-8a2b-c37724816c8c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9599efe61953407b827ce2a621426633'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:27:59.187426 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 831/1672] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 24 09:27:58 2022] DELETE /identity/v3/users/9599efe61953407b827ce2a621426633 => generated 0 bytes in 589 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:27:59.998863 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c47d11-be96-4c1f-83eb-7e3b424ccaed admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:28:00.000476 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-16c47d11-be96-4c1f-83eb-7e3b424ccaed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:28:00.039853 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c47d11-be96-4c1f-83eb-7e3b424ccaed admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:28:00.230130 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46aee2d-4424-4398-8d69-b471587619f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:28:00.230791 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46aee2d-4424-4398-8d69-b471587619f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:00.231070 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46aee2d-4424-4398-8d69-b471587619f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:00.231344 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46aee2d-4424-4398-8d69-b471587619f2 admin admin] PATH_INFO: `/v3/projects/1a132169db34442cb6e7f2ba7cf94962` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:00.240643 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46aee2d-4424-4398-8d69-b471587619f2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1a132169db34442cb6e7f2ba7cf94962)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:28:00.243122 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46aee2d-4424-4398-8d69-b471587619f2 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:28:00.283930 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2e27d2-c94c-481a-8a2b-c37724816c8c admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:28:00.285774 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-2d2e27d2-c94c-481a-8a2b-c37724816c8c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:28:00.324950 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2e27d2-c94c-481a-8a2b-c37724816c8c admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:28:00.349793 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d15752e-2818-4cff-93d3-257e62c9c0c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:28:00.350669 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d15752e-2818-4cff-93d3-257e62c9c0c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:00.351091 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d15752e-2818-4cff-93d3-257e62c9c0c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:00.351503 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d15752e-2818-4cff-93d3-257e62c9c0c8 admin admin] PATH_INFO: `/v3/projects/c23aab1ccdd9406da39a2d3302a81be3` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:00.354665 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d15752e-2818-4cff-93d3-257e62c9c0c8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c23aab1ccdd9406da39a2d3302a81be3)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:28:00.356868 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d15752e-2818-4cff-93d3-257e62c9c0c8 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:28:00.382517 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:28:00.465710 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-d46aee2d-4424-4398-8d69-b471587619f2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1a132169db34442cb6e7f2ba7cf94962', 'request_id': 'req-d46aee2d-4424-4398-8d69-b471587619f2'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:28:00.480873 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-d46aee2d-4424-4398-8d69-b471587619f2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1a132169db34442cb6e7f2ba7cf94962', 'request_id': 'req-d46aee2d-4424-4398-8d69-b471587619f2'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:28:00.482770 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-8d15752e-2818-4cff-93d3-257e62c9c0c8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c23aab1ccdd9406da39a2d3302a81be3', 'request_id': 'req-8d15752e-2818-4cff-93d3-257e62c9c0c8'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:28:00.499394 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-8d15752e-2818-4cff-93d3-257e62c9c0c8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c23aab1ccdd9406da39a2d3302a81be3', 'request_id': 'req-8d15752e-2818-4cff-93d3-257e62c9c0c8'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:28:00.502301 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-d46aee2d-4424-4398-8d69-b471587619f2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1a132169db34442cb6e7f2ba7cf94962', 'request_id': 'req-d46aee2d-4424-4398-8d69-b471587619f2'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:28:00.524754 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-8d15752e-2818-4cff-93d3-257e62c9c0c8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c23aab1ccdd9406da39a2d3302a81be3', 'request_id': 'req-8d15752e-2818-4cff-93d3-257e62c9c0c8'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:28:00.590002 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-d46aee2d-4424-4398-8d69-b471587619f2 admin admin] The token cache is being invalidate because project 1a132169db34442cb6e7f2ba7cf94962 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:28:00.590870 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-d46aee2d-4424-4398-8d69-b471587619f2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:28:00.591434 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-d46aee2d-4424-4398-8d69-b471587619f2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:28:00.594038 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 842/1673] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 24 09:27:59 2022] DELETE /identity/v3/projects/1a132169db34442cb6e7f2ba7cf94962 => generated 0 bytes in 596 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:28:00.596553 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe68e3b-99cd-4c76-9216-6d8e0b7bdcbc None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:00.596845 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe68e3b-99cd-4c76-9216-6d8e0b7bdcbc None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:00.597276 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe68e3b-99cd-4c76-9216-6d8e0b7bdcbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:00.648146 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-8d15752e-2818-4cff-93d3-257e62c9c0c8 admin admin] The token cache is being invalidate because project c23aab1ccdd9406da39a2d3302a81be3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81602) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:28:00.648387 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-8d15752e-2818-4cff-93d3-257e62c9c0c8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:28:00.649566 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-8d15752e-2818-4cff-93d3-257e62c9c0c8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:28:00.657373 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 832/1674] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 24 09:28:00 2022] DELETE /identity/v3/projects/c23aab1ccdd9406da39a2d3302a81be3 => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:28:00.658928 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3621f133-2850-4d06-82f1-763750834f9c None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:00.659401 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3621f133-2850-4d06-82f1-763750834f9c None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:00.659854 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3621f133-2850-4d06-82f1-763750834f9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:00.701844 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-efe68e3b-99cd-4c76-9216-6d8e0b7bdcbc None None] MFA Rules not processed for user `def04335e57e48409a6943fa231d47c3`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:28:00.703630 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-efe68e3b-99cd-4c76-9216-6d8e0b7bdcbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:28:00.704844 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-3621f133-2850-4d06-82f1-763750834f9c None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:28:00.706172 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-3621f133-2850-4d06-82f1-763750834f9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:28:00.856014 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 833/1675] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 09:28:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:28:00.858902 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2f49be-927c-4cad-9e06-5dbffe91a4cc None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:00.864421 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2f49be-927c-4cad-9e06-5dbffe91a4cc None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:00.864974 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2f49be-927c-4cad-9e06-5dbffe91a4cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:00.912328 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-ef2f49be-927c-4cad-9e06-5dbffe91a4cc None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:28:00.914165 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-ef2f49be-927c-4cad-9e06-5dbffe91a4cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:28:00.938025 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 834/1676] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 09:28:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:28:00.943443 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38b6603-72bd-4727-bbe4-51a5f52ec9f1 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:00.944910 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38b6603-72bd-4727-bbe4-51a5f52ec9f1 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:00.948288 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38b6603-72bd-4727-bbe4-51a5f52ec9f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:00.974615 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:28:00.980197 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 843/1677] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 09:28:00 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 386 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:28:00.983567 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75616043-8aea-428e-bce4-4af6072dcc02 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:00.984093 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75616043-8aea-428e-bce4-4af6072dcc02 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:00.984578 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75616043-8aea-428e-bce4-4af6072dcc02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:00.992871 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-e38b6603-72bd-4727-bbe4-51a5f52ec9f1 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:28:00.994184 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-e38b6603-72bd-4727-bbe4-51a5f52ec9f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:28:01.016386 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-75616043-8aea-428e-bce4-4af6072dcc02 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:28:01.020953 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-75616043-8aea-428e-bce4-4af6072dcc02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:28:01.021202 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 835/1678] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 09:28:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:28:01.021202 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38b6603-72bd-4727-bbe4-51a5f52ec9f1 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:28:01.027617 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-e38b6603-72bd-4727-bbe4-51a5f52ec9f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:28:01.037818 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 844/1679] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 09:28:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:28:01.039372 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75616043-8aea-428e-bce4-4af6072dcc02 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:28:01.073097 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75616043-8aea-428e-bce4-4af6072dcc02 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:28:01.080640 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef94b5f-753d-43b8-97a2-5039f7808dee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:28:01.082982 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef94b5f-753d-43b8-97a2-5039f7808dee admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:01.084566 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef94b5f-753d-43b8-97a2-5039f7808dee admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:01.085731 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef94b5f-753d-43b8-97a2-5039f7808dee admin admin] PATH_INFO: `/v3/domains` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:01.087212 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef94b5f-753d-43b8-97a2-5039f7808dee admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:28:01.089845 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef94b5f-753d-43b8-97a2-5039f7808dee admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:28:01.098301 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef94b5f-753d-43b8-97a2-5039f7808dee admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:28:01.147644 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 845/1680] 10.222.0.16 () {58 vars in 1153 bytes} [Wed Aug 24 09:28:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:28:01.147795 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38b6603-72bd-4727-bbe4-51a5f52ec9f1 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:28:01.154205 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef94b5f-753d-43b8-97a2-5039f7808dee admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:28:01.161566 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fe960e-68af-4d4b-9135-5d05173cb374 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f055f7dc51d43bda9b315ca67f11d58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:28:01.162212 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fe960e-68af-4d4b-9135-5d05173cb374 service nova] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:01.162665 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fe960e-68af-4d4b-9135-5d05173cb374 service nova] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:01.163480 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fe960e-68af-4d4b-9135-5d05173cb374 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:01.173450 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fe960e-68af-4d4b-9135-5d05173cb374 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:28:01.180360 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fe960e-68af-4d4b-9135-5d05173cb374 service nova] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:28:01.198870 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef94b5f-753d-43b8-97a2-5039f7808dee admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:28:01.199142 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 836/1681] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:28:01 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:28:01.200499 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fe960e-68af-4d4b-9135-5d05173cb374 service nova] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:28:01.212710 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fe960e-68af-4d4b-9135-5d05173cb374 service nova] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:28:01.218175 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f31fd4-7db2-4894-a647-bad6fb5c0c7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:28:01.218776 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f31fd4-7db2-4894-a647-bad6fb5c0c7b admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:01.219116 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f31fd4-7db2-4894-a647-bad6fb5c0c7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:01.219395 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f31fd4-7db2-4894-a647-bad6fb5c0c7b admin admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:01.220358 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f31fd4-7db2-4894-a647-bad6fb5c0c7b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:28:01.220671 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95532411-4e3f-42f0-8823-95c1dd6039e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:28:01.222440 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f31fd4-7db2-4894-a647-bad6fb5c0c7b admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:28:01.224132 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95532411-4e3f-42f0-8823-95c1dd6039e0 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:01.224946 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95532411-4e3f-42f0-8823-95c1dd6039e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:01.231905 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95532411-4e3f-42f0-8823-95c1dd6039e0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:01.235225 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95532411-4e3f-42f0-8823-95c1dd6039e0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:28:01.237195 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95532411-4e3f-42f0-8823-95c1dd6039e0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:28:01.243493 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95532411-4e3f-42f0-8823-95c1dd6039e0 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:28:01.308033 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 846/1682] 10.222.0.16 () {58 vars in 1153 bytes} [Wed Aug 24 09:28:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:28:01.309114 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95532411-4e3f-42f0-8823-95c1dd6039e0 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:28:01.340050 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95532411-4e3f-42f0-8823-95c1dd6039e0 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:28:01.364560 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eff13bd-6f73-4bdd-be98-04a8b2826642 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:28:01.365823 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eff13bd-6f73-4bdd-be98-04a8b2826642 admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:01.367361 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eff13bd-6f73-4bdd-be98-04a8b2826642 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:01.367897 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eff13bd-6f73-4bdd-be98-04a8b2826642 admin admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:01.368761 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eff13bd-6f73-4bdd-be98-04a8b2826642 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:28:01.373432 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eff13bd-6f73-4bdd-be98-04a8b2826642 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:28:01.386586 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 837/1683] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 24 09:28:01 2022] POST /identity/v3/projects => generated 363 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:28:01.392889 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f31fd4-7db2-4894-a647-bad6fb5c0c7b admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:28:01.525145 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 847/1684] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 24 09:28:01 2022] POST /identity/v3/projects => generated 357 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:28:01.528907 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eff13bd-6f73-4bdd-be98-04a8b2826642 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:28:01.542698 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f31fd4-7db2-4894-a647-bad6fb5c0c7b admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:28:01.671805 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fb2049-dea7-42fa-9553-bb84016eedfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:28:01.672682 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fb2049-dea7-42fa-9553-bb84016eedfd admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:01.673307 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fb2049-dea7-42fa-9553-bb84016eedfd admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:01.673916 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fb2049-dea7-42fa-9553-bb84016eedfd admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:01.674748 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fb2049-dea7-42fa-9553-bb84016eedfd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:28:01.677739 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fb2049-dea7-42fa-9553-bb84016eedfd admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:28:01.714496 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eff13bd-6f73-4bdd-be98-04a8b2826642 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:28:01.720166 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a289d2-f391-4f89-a6d7-542b4f82b16a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:28:01.720874 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a289d2-f391-4f89-a6d7-542b4f82b16a admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:01.721160 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a289d2-f391-4f89-a6d7-542b4f82b16a admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:01.721432 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a289d2-f391-4f89-a6d7-542b4f82b16a admin admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:01.722070 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a289d2-f391-4f89-a6d7-542b4f82b16a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:28:01.724226 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a289d2-f391-4f89-a6d7-542b4f82b16a admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:28:01.839389 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 848/1685] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 24 09:28:01 2022] POST /identity/v3/users => generated 279 bytes in 308 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 09:28:01.841523 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:28:01.843442 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a289d2-f391-4f89-a6d7-542b4f82b16a admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:28:01.844910 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 838/1686] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 24 09:28:01 2022] POST /identity/v3/users => generated 282 bytes in 453 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:28:01.846308 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fb2049-dea7-42fa-9553-bb84016eedfd admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:28:01.859673 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a289d2-f391-4f89-a6d7-542b4f82b16a admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:28:01.876537 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34cf937-30f0-4edf-86fe-ebc5b94b7e87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:28:01.877294 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34cf937-30f0-4edf-86fe-ebc5b94b7e87 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:01.877579 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34cf937-30f0-4edf-86fe-ebc5b94b7e87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:01.878028 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34cf937-30f0-4edf-86fe-ebc5b94b7e87 admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:01.878699 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34cf937-30f0-4edf-86fe-ebc5b94b7e87 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:28:01.880385 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34cf937-30f0-4edf-86fe-ebc5b94b7e87 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:28:01.891333 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fb2049-dea7-42fa-9553-bb84016eedfd admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:28:01.907002 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba3892b-97ee-40de-a8a2-60accf9370f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:28:01.907788 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba3892b-97ee-40de-a8a2-60accf9370f7 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:01.908237 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba3892b-97ee-40de-a8a2-60accf9370f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:01.909599 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba3892b-97ee-40de-a8a2-60accf9370f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:01.910428 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba3892b-97ee-40de-a8a2-60accf9370f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:28:01.914462 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba3892b-97ee-40de-a8a2-60accf9370f7 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:28:01.919979 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 849/1687] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 24 09:28:01 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 09:28:01.925254 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34cf937-30f0-4edf-86fe-ebc5b94b7e87 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:28:01.948444 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 839/1688] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 24 09:28:01 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:28:01.949557 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba3892b-97ee-40de-a8a2-60accf9370f7 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:28:01.962774 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34cf937-30f0-4edf-86fe-ebc5b94b7e87 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:28:01.963198 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba3892b-97ee-40de-a8a2-60accf9370f7 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:28:01.969213 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cfbc9f-42c7-4f90-963b-d94fb76f8150 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:28:01.970245 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cfbc9f-42c7-4f90-963b-d94fb76f8150 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:01.970595 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cfbc9f-42c7-4f90-963b-d94fb76f8150 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:01.970887 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cfbc9f-42c7-4f90-963b-d94fb76f8150 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:01.971658 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cfbc9f-42c7-4f90-963b-d94fb76f8150 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:28:01.973810 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cfbc9f-42c7-4f90-963b-d94fb76f8150 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:28:01.979757 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57e4116-9740-4a37-9a4f-1fee74bc7701 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:28:01.980378 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57e4116-9740-4a37-9a4f-1fee74bc7701 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:01.981079 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57e4116-9740-4a37-9a4f-1fee74bc7701 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:01.981368 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57e4116-9740-4a37-9a4f-1fee74bc7701 admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:01.982192 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57e4116-9740-4a37-9a4f-1fee74bc7701 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:28:01.983685 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57e4116-9740-4a37-9a4f-1fee74bc7701 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:28:01.998798 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 840/1689] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 24 09:28:01 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:28:02.000249 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cfbc9f-42c7-4f90-963b-d94fb76f8150 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:28:02.012246 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cfbc9f-42c7-4f90-963b-d94fb76f8150 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:28:02.018309 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f23bc6-2e3c-4e19-9771-55ca4d975669 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:28:02.020629 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f23bc6-2e3c-4e19-9771-55ca4d975669 admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:02.020629 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f23bc6-2e3c-4e19-9771-55ca4d975669 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:02.020629 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f23bc6-2e3c-4e19-9771-55ca4d975669 admin admin] PATH_INFO: `/v3/projects/f613b5ff3b6649868a283bc063e9a978/users/30d814217f074fd7a57cef02ceddc31f/roles/136cdf78e00a426997dd4e37faff489b` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:02.029820 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 850/1690] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 24 09:28:01 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:28:02.031016 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57e4116-9740-4a37-9a4f-1fee74bc7701 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:28:02.046036 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f23bc6-2e3c-4e19-9771-55ca4d975669 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f613b5ff3b6649868a283bc063e9a978, user_id=30d814217f074fd7a57cef02ceddc31f, role_id=136cdf78e00a426997dd4e37faff489b)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:28:02.051107 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f23bc6-2e3c-4e19-9771-55ca4d975669 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:28:02.054270 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57e4116-9740-4a37-9a4f-1fee74bc7701 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:28:02.081289 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fbd089-3abc-432b-9cf6-d33fa2383e0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:28:02.084143 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fbd089-3abc-432b-9cf6-d33fa2383e0a admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:02.090807 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fbd089-3abc-432b-9cf6-d33fa2383e0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:02.090807 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fbd089-3abc-432b-9cf6-d33fa2383e0a admin admin] PATH_INFO: `/v3/projects/820e170187a442a99dc2e348cf0fc4bc/users/cb4f9bad649d44a9aa6687f00116f9b3/roles/136cdf78e00a426997dd4e37faff489b` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:02.130813 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fbd089-3abc-432b-9cf6-d33fa2383e0a admin admin] RBAC: Authorizing `identity:create_grant(project_id=820e170187a442a99dc2e348cf0fc4bc, user_id=cb4f9bad649d44a9aa6687f00116f9b3, role_id=136cdf78e00a426997dd4e37faff489b)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:28:02.139991 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fbd089-3abc-432b-9cf6-d33fa2383e0a admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:28:02.150108 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 841/1691] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 24 09:28:01 2022] PUT /identity/v3/projects/f613b5ff3b6649868a283bc063e9a978/users/30d814217f074fd7a57cef02ceddc31f/roles/136cdf78e00a426997dd4e37faff489b => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:28:02.165244 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dc20ea-b5dc-4ab2-85a4-50edc4089dbc None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:02.167963 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dc20ea-b5dc-4ab2-85a4-50edc4089dbc None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:02.168265 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dc20ea-b5dc-4ab2-85a4-50edc4089dbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:02.248545 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 851/1692] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 24 09:28:02 2022] PUT /identity/v3/projects/820e170187a442a99dc2e348cf0fc4bc/users/cb4f9bad649d44a9aa6687f00116f9b3/roles/136cdf78e00a426997dd4e37faff489b => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:28:02.257354 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-60dc20ea-b5dc-4ab2-85a4-50edc4089dbc None None] MFA Rules not processed for user `30d814217f074fd7a57cef02ceddc31f`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:28:02.257354 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-60dc20ea-b5dc-4ab2-85a4-50edc4089dbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:28:02.258796 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c286c58a-f75e-433e-89fa-663b8b00fbd7 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:02.259092 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c286c58a-f75e-433e-89fa-663b8b00fbd7 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:02.259377 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c286c58a-f75e-433e-89fa-663b8b00fbd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:02.378139 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-c286c58a-f75e-433e-89fa-663b8b00fbd7 None None] MFA Rules not processed for user `cb4f9bad649d44a9aa6687f00116f9b3`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:28:02.381562 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-c286c58a-f75e-433e-89fa-663b8b00fbd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:28:02.389053 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 842/1693] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 09:28:02 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:28:02.390614 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dc20ea-b5dc-4ab2-85a4-50edc4089dbc None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:28:02.396559 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-60dc20ea-b5dc-4ab2-85a4-50edc4089dbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:28:02.437289 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:28:02.482300 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 852/1694] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 09:28:02 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 09:28:02.483285 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c286c58a-f75e-433e-89fa-663b8b00fbd7 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:28:02.525697 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dc20ea-b5dc-4ab2-85a4-50edc4089dbc None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:28:02.610417 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2348c035-e6ab-496f-a1ec-33beceeec986 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:28:02.610417 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2348c035-e6ab-496f-a1ec-33beceeec986 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:02.610417 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2348c035-e6ab-496f-a1ec-33beceeec986 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:02.610417 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2348c035-e6ab-496f-a1ec-33beceeec986 admin admin] PATH_INFO: `/v3/users/e93fc592784e4dd7b22e8fa596c6cee5` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:02.613741 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2348c035-e6ab-496f-a1ec-33beceeec986 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e93fc592784e4dd7b22e8fa596c6cee5)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:28:02.613741 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2348c035-e6ab-496f-a1ec-33beceeec986 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:28:02.631470 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c286c58a-f75e-433e-89fa-663b8b00fbd7 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:28:02.640634 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030b5658-2317-4d62-8128-5529153949f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:28:02.642246 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030b5658-2317-4d62-8128-5529153949f4 admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:02.642874 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030b5658-2317-4d62-8128-5529153949f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:02.643883 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030b5658-2317-4d62-8128-5529153949f4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:02.647138 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030b5658-2317-4d62-8128-5529153949f4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:28:02.663227 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030b5658-2317-4d62-8128-5529153949f4 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:28:02.749970 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-2348c035-e6ab-496f-a1ec-33beceeec986 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e93fc592784e4dd7b22e8fa596c6cee5'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:28:02.751298 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-2348c035-e6ab-496f-a1ec-33beceeec986 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e93fc592784e4dd7b22e8fa596c6cee5'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:28:02.783053 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 853/1695] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 24 09:28:02 2022] POST /identity/v3/projects => generated 365 bytes in 300 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:28:02.785028 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030b5658-2317-4d62-8128-5529153949f4 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:28:02.817143 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-2348c035-e6ab-496f-a1ec-33beceeec986 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e93fc592784e4dd7b22e8fa596c6cee5'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:28:02.818570 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-2348c035-e6ab-496f-a1ec-33beceeec986 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e93fc592784e4dd7b22e8fa596c6cee5'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:28:02.830764 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-2348c035-e6ab-496f-a1ec-33beceeec986 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e93fc592784e4dd7b22e8fa596c6cee5'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:28:02.881261 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 843/1696] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 24 09:28:02 2022] DELETE /identity/v3/users/e93fc592784e4dd7b22e8fa596c6cee5 => generated 0 bytes in 491 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:28:02.882562 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2348c035-e6ab-496f-a1ec-33beceeec986 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:28:02.885304 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-2348c035-e6ab-496f-a1ec-33beceeec986 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:28:02.960212 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030b5658-2317-4d62-8128-5529153949f4 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:28:03.004919 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2348c035-e6ab-496f-a1ec-33beceeec986 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:28:03.011811 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f37696f-1eab-43ba-9153-8df64395bf1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:28:03.012568 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f37696f-1eab-43ba-9153-8df64395bf1c admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:03.012952 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f37696f-1eab-43ba-9153-8df64395bf1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:03.013325 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f37696f-1eab-43ba-9153-8df64395bf1c admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:03.014186 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f37696f-1eab-43ba-9153-8df64395bf1c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:28:03.027394 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f37696f-1eab-43ba-9153-8df64395bf1c admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:28:03.080447 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:28:03.119874 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 844/1697] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 24 09:28:02 2022] POST /identity/v3/users => generated 289 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:28:03.119874 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf81ed5-12e8-4e26-b4a5-8a2477b2ee0c None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:03.119874 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf81ed5-12e8-4e26-b4a5-8a2477b2ee0c None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:03.119874 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf81ed5-12e8-4e26-b4a5-8a2477b2ee0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:03.123168 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4319c49-ce9e-4977-8d05-78ca278ebf38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:28:03.123867 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4319c49-ce9e-4977-8d05-78ca278ebf38 admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:03.124102 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4319c49-ce9e-4977-8d05-78ca278ebf38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:03.124552 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4319c49-ce9e-4977-8d05-78ca278ebf38 admin admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:03.125351 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4319c49-ce9e-4977-8d05-78ca278ebf38 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:28:03.127462 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4319c49-ce9e-4977-8d05-78ca278ebf38 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:28:03.177972 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-6bf81ed5-12e8-4e26-b4a5-8a2477b2ee0c None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:28:03.179280 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-6bf81ed5-12e8-4e26-b4a5-8a2477b2ee0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:28:03.199648 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 845/1698] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 09:28:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:28:03.205854 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf81ed5-12e8-4e26-b4a5-8a2477b2ee0c None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:28:03.220076 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf81ed5-12e8-4e26-b4a5-8a2477b2ee0c None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:28:03.229088 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ba1b74-e8b4-40be-940c-2f353aaf5109 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:28:03.229888 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ba1b74-e8b4-40be-940c-2f353aaf5109 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:03.230291 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ba1b74-e8b4-40be-940c-2f353aaf5109 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:03.230635 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ba1b74-e8b4-40be-940c-2f353aaf5109 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:03.231424 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ba1b74-e8b4-40be-940c-2f353aaf5109 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:28:03.233257 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ba1b74-e8b4-40be-940c-2f353aaf5109 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:28:03.250408 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 846/1699] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 24 09:28:03 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:28:03.251731 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ba1b74-e8b4-40be-940c-2f353aaf5109 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:28:03.253864 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-58ba1b74-e8b4-40be-940c-2f353aaf5109 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:28:03.260377 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 854/1700] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 24 09:28:02 2022] POST /identity/v3/projects => generated 355 bytes in 477 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:28:03.267193 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4319c49-ce9e-4977-8d05-78ca278ebf38 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:28:03.370359 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:28:03.425814 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ba1b74-e8b4-40be-940c-2f353aaf5109 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:28:03.454559 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4319c49-ce9e-4977-8d05-78ca278ebf38 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:28:03.473312 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772a5889-6bd0-4885-9348-9b789c96d9ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:28:03.474033 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772a5889-6bd0-4885-9348-9b789c96d9ca admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:03.474317 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772a5889-6bd0-4885-9348-9b789c96d9ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:03.474593 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772a5889-6bd0-4885-9348-9b789c96d9ca admin admin] PATH_INFO: `/v3/projects/44a12461a70846a7aafc03f342c074a3/users/a87941c110c049c4becad18bf3b09d94/roles/15f634d4a2d247ddab90c458e5e1c141` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:03.523891 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772a5889-6bd0-4885-9348-9b789c96d9ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=44a12461a70846a7aafc03f342c074a3, user_id=a87941c110c049c4becad18bf3b09d94, role_id=15f634d4a2d247ddab90c458e5e1c141)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:28:03.531789 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772a5889-6bd0-4885-9348-9b789c96d9ca admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:28:03.561199 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19500eec-b286-4305-8c59-c4be35cf9190 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e5260e798404e8bb12b25ca8ea97052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:28:03.561845 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19500eec-b286-4305-8c59-c4be35cf9190 service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:03.562239 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19500eec-b286-4305-8c59-c4be35cf9190 service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:03.562417 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19500eec-b286-4305-8c59-c4be35cf9190 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:03.584077 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19500eec-b286-4305-8c59-c4be35cf9190 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:28:03.595320 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19500eec-b286-4305-8c59-c4be35cf9190 service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:28:03.617896 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 855/1701] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 24 09:28:03 2022] PUT /identity/v3/projects/44a12461a70846a7aafc03f342c074a3/users/a87941c110c049c4becad18bf3b09d94/roles/15f634d4a2d247ddab90c458e5e1c141 => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:28:03.628825 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772a5889-6bd0-4885-9348-9b789c96d9ca admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:28:03.628825 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-772a5889-6bd0-4885-9348-9b789c96d9ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:28:03.629040 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 847/1702] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:28:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:28:03.629040 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19500eec-b286-4305-8c59-c4be35cf9190 service neutron] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:28:03.645901 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19500eec-b286-4305-8c59-c4be35cf9190 service neutron] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:28:03.761965 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87fe51e-d684-4807-8ba4-f02512daf7f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:28:03.762599 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87fe51e-d684-4807-8ba4-f02512daf7f3 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:03.762959 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87fe51e-d684-4807-8ba4-f02512daf7f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:03.763151 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87fe51e-d684-4807-8ba4-f02512daf7f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:03.764349 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87fe51e-d684-4807-8ba4-f02512daf7f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:28:03.766354 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87fe51e-d684-4807-8ba4-f02512daf7f3 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:28:03.788078 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 848/1703] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 24 09:28:03 2022] GET /identity/v3/roles => generated 1420 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:28:03.789016 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87fe51e-d684-4807-8ba4-f02512daf7f3 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:28:03.803477 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87fe51e-d684-4807-8ba4-f02512daf7f3 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:28:03.810381 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0ffc98-6f59-46aa-8af6-9f508c92201c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:28:03.811152 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0ffc98-6f59-46aa-8af6-9f508c92201c admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:03.811712 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0ffc98-6f59-46aa-8af6-9f508c92201c admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:03.812277 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0ffc98-6f59-46aa-8af6-9f508c92201c admin admin] PATH_INFO: `/v3/projects/44a12461a70846a7aafc03f342c074a3/users/a87941c110c049c4becad18bf3b09d94/roles/15f634d4a2d247ddab90c458e5e1c141` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:03.814421 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:28:03.816338 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0ffc98-6f59-46aa-8af6-9f508c92201c admin admin] RBAC: Authorizing `identity:create_grant(project_id=44a12461a70846a7aafc03f342c074a3, user_id=a87941c110c049c4becad18bf3b09d94, role_id=15f634d4a2d247ddab90c458e5e1c141)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:28:03.822657 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0ffc98-6f59-46aa-8af6-9f508c92201c admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:28:03.832265 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772a5889-6bd0-4885-9348-9b789c96d9ca admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:28:03.840353 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 849/1704] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 24 09:28:03 2022] PUT /identity/v3/projects/44a12461a70846a7aafc03f342c074a3/users/a87941c110c049c4becad18bf3b09d94/roles/15f634d4a2d247ddab90c458e5e1c141 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:28:03.846798 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c8a196-f23e-4049-ac06-7ef3942ad54b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:28:03.847444 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c8a196-f23e-4049-ac06-7ef3942ad54b admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:03.847783 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c8a196-f23e-4049-ac06-7ef3942ad54b admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:03.848065 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c8a196-f23e-4049-ac06-7ef3942ad54b admin admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:03.848703 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c8a196-f23e-4049-ac06-7ef3942ad54b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:28:03.854087 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40232bee-6c35-47d1-8244-d36f69c20de7 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:03.854448 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40232bee-6c35-47d1-8244-d36f69c20de7 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:03.854794 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40232bee-6c35-47d1-8244-d36f69c20de7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:03.855617 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c8a196-f23e-4049-ac06-7ef3942ad54b admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:28:03.893885 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-40232bee-6c35-47d1-8244-d36f69c20de7 None None] MFA Rules not processed for user `a87941c110c049c4becad18bf3b09d94`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:28:03.898440 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-40232bee-6c35-47d1-8244-d36f69c20de7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:28:03.966485 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 856/1705] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 24 09:28:03 2022] POST /identity/v3/users => generated 284 bytes in 348 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:28:03.968270 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421c3cfc-0535-4fb3-884f-81082dd1b5c8 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:03.968558 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421c3cfc-0535-4fb3-884f-81082dd1b5c8 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:03.968836 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421c3cfc-0535-4fb3-884f-81082dd1b5c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:03.988318 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 850/1706] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 09:28:03 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:28:03.989413 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40232bee-6c35-47d1-8244-d36f69c20de7 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:28:04.031397 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-421c3cfc-0535-4fb3-884f-81082dd1b5c8 None None] MFA Rules not processed for user `6cad69b6035948fe91e9beb66c807d28`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:28:04.032897 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-421c3cfc-0535-4fb3-884f-81082dd1b5c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:28:04.059660 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40232bee-6c35-47d1-8244-d36f69c20de7 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:28:04.071743 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e6e634-c3e5-4977-9abb-19999ec8c26d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:28:04.072765 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e6e634-c3e5-4977-9abb-19999ec8c26d admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:04.073276 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e6e634-c3e5-4977-9abb-19999ec8c26d admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:04.074351 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e6e634-c3e5-4977-9abb-19999ec8c26d admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:04.077970 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e6e634-c3e5-4977-9abb-19999ec8c26d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:28:04.080338 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e6e634-c3e5-4977-9abb-19999ec8c26d admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:28:04.107997 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 851/1707] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 24 09:28:03 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:28:04.109413 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e6e634-c3e5-4977-9abb-19999ec8c26d admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:28:04.117035 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-a1e6e634-c3e5-4977-9abb-19999ec8c26d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:28:04.130172 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:28:04.147657 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 857/1708] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 09:28:03 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:28:04.147657 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421c3cfc-0535-4fb3-884f-81082dd1b5c8 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:28:04.169056 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421c3cfc-0535-4fb3-884f-81082dd1b5c8 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:28:04.186869 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323db549-19c8-4997-a423-5c24f9c3cb6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:28:04.188166 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323db549-19c8-4997-a423-5c24f9c3cb6b admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:04.188615 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323db549-19c8-4997-a423-5c24f9c3cb6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:04.189295 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323db549-19c8-4997-a423-5c24f9c3cb6b admin admin] PATH_INFO: `/v3/projects/bcb0ac58da8d49559c4c24dff72e54fd/users/1ee501ef4b9e4233a415891ce2c68f3d/roles/15f634d4a2d247ddab90c458e5e1c141` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:04.229738 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323db549-19c8-4997-a423-5c24f9c3cb6b admin admin] RBAC: Authorizing `identity:create_grant(project_id=bcb0ac58da8d49559c4c24dff72e54fd, user_id=1ee501ef4b9e4233a415891ce2c68f3d, role_id=15f634d4a2d247ddab90c458e5e1c141)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:28:04.241434 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323db549-19c8-4997-a423-5c24f9c3cb6b admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:28:04.244868 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e6e634-c3e5-4977-9abb-19999ec8c26d admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:28:04.257545 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14df8dc2-0211-4716-b05d-161b8db5e952 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c32b83577c46488906fe409a790b24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:28:04.258362 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14df8dc2-0211-4716-b05d-161b8db5e952 service cinder] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:04.258646 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14df8dc2-0211-4716-b05d-161b8db5e952 service cinder] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:04.258921 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14df8dc2-0211-4716-b05d-161b8db5e952 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:04.279114 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14df8dc2-0211-4716-b05d-161b8db5e952 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:28:04.281877 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14df8dc2-0211-4716-b05d-161b8db5e952 service cinder] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:28:04.300588 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 852/1709] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:28:04 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:28:04.302051 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0504d7-21e9-4077-b65b-208d6e8a6cf6 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:04.302343 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0504d7-21e9-4077-b65b-208d6e8a6cf6 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:04.302634 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0504d7-21e9-4077-b65b-208d6e8a6cf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:04.333668 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 858/1710] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 24 09:28:04 2022] PUT /identity/v3/projects/bcb0ac58da8d49559c4c24dff72e54fd/users/1ee501ef4b9e4233a415891ce2c68f3d/roles/15f634d4a2d247ddab90c458e5e1c141 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:28:04.337659 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323db549-19c8-4997-a423-5c24f9c3cb6b admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:28:04.363758 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-4a0504d7-21e9-4077-b65b-208d6e8a6cf6 None None] MFA Rules not processed for user `6cad69b6035948fe91e9beb66c807d28`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:28:04.365003 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-4a0504d7-21e9-4077-b65b-208d6e8a6cf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:28:04.440601 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:28:04.508410 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323db549-19c8-4997-a423-5c24f9c3cb6b admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:28:04.511905 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 853/1711] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 09:28:04 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:28:04.525077 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0504d7-21e9-4077-b65b-208d6e8a6cf6 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:28:04.527023 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-4a0504d7-21e9-4077-b65b-208d6e8a6cf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:28:04.527184 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f75aed-d531-4c85-a298-7194536d16ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:28:04.527914 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f75aed-d531-4c85-a298-7194536d16ec admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:04.528279 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f75aed-d531-4c85-a298-7194536d16ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:04.528453 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f75aed-d531-4c85-a298-7194536d16ec admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:04.529165 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f75aed-d531-4c85-a298-7194536d16ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:28:04.531615 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f75aed-d531-4c85-a298-7194536d16ec admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:28:04.561400 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 859/1712] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 24 09:28:04 2022] GET /identity/v3/roles => generated 1420 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:28:04.562378 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f75aed-d531-4c85-a298-7194536d16ec admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:28:04.587095 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f75aed-d531-4c85-a298-7194536d16ec admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:28:04.602534 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8f991b-2f36-4b2b-bedd-9966cdd49338 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:28:04.603167 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8f991b-2f36-4b2b-bedd-9966cdd49338 admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:04.603448 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8f991b-2f36-4b2b-bedd-9966cdd49338 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:04.603793 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8f991b-2f36-4b2b-bedd-9966cdd49338 admin admin] PATH_INFO: `/v3/projects/bcb0ac58da8d49559c4c24dff72e54fd/users/1ee501ef4b9e4233a415891ce2c68f3d/roles/15f634d4a2d247ddab90c458e5e1c141` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:04.608015 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8f991b-2f36-4b2b-bedd-9966cdd49338 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bcb0ac58da8d49559c4c24dff72e54fd, user_id=1ee501ef4b9e4233a415891ce2c68f3d, role_id=15f634d4a2d247ddab90c458e5e1c141)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:28:04.613382 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8f991b-2f36-4b2b-bedd-9966cdd49338 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:28:04.644350 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 860/1713] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 24 09:28:04 2022] PUT /identity/v3/projects/bcb0ac58da8d49559c4c24dff72e54fd/users/1ee501ef4b9e4233a415891ce2c68f3d/roles/15f634d4a2d247ddab90c458e5e1c141 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:28:04.645985 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8f991b-2f36-4b2b-bedd-9966cdd49338 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:28:04.704450 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0504d7-21e9-4077-b65b-208d6e8a6cf6 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:28:04.783603 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8f991b-2f36-4b2b-bedd-9966cdd49338 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:28:04.796243 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbdf321-c778-4932-b901-33803373f2f4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c32b83577c46488906fe409a790b24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:28:04.796243 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbdf321-c778-4932-b901-33803373f2f4 service cinder] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:04.796243 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbdf321-c778-4932-b901-33803373f2f4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:04.796243 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbdf321-c778-4932-b901-33803373f2f4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:04.797823 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-0bbdf321-c778-4932-b901-33803373f2f4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:28:04.912968 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a1ddb0-7a34-415c-8f4b-164735c37074 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f055f7dc51d43bda9b315ca67f11d58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:28:04.914595 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a1ddb0-7a34-415c-8f4b-164735c37074 service nova] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:04.916795 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a1ddb0-7a34-415c-8f4b-164735c37074 service nova] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:04.916956 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a1ddb0-7a34-415c-8f4b-164735c37074 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:04.937201 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbdf321-c778-4932-b901-33803373f2f4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:28:04.939083 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbdf321-c778-4932-b901-33803373f2f4 service cinder] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:28:04.948992 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:28:04.952127 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 861/1714] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:28:04 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:28:04.953050 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbdf321-c778-4932-b901-33803373f2f4 service cinder] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:28:04.954376 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-0bbdf321-c778-4932-b901-33803373f2f4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:28:04.979627 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:28:05.059044 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a1ddb0-7a34-415c-8f4b-164735c37074 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:28:05.062186 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a1ddb0-7a34-415c-8f4b-164735c37074 service nova] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:28:05.077483 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 854/1715] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:28:04 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 553 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 09:28:05.081168 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2727f329-05b5-4fab-b0f3-eeafde6afe92 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:05.087480 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2727f329-05b5-4fab-b0f3-eeafde6afe92 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:05.088636 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2727f329-05b5-4fab-b0f3-eeafde6afe92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:05.127723 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-2727f329-05b5-4fab-b0f3-eeafde6afe92 None None] MFA Rules not processed for user `1ee501ef4b9e4233a415891ce2c68f3d`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:28:05.128406 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbdf321-c778-4932-b901-33803373f2f4 service cinder] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:28:05.129213 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-2727f329-05b5-4fab-b0f3-eeafde6afe92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:28:05.135702 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8122b3f6-dfad-48da-bf70-f6644538707f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:28:05.136365 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8122b3f6-dfad-48da-bf70-f6644538707f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:05.136649 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8122b3f6-dfad-48da-bf70-f6644538707f admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:05.136926 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8122b3f6-dfad-48da-bf70-f6644538707f admin admin] PATH_INFO: `/v3/projects/6a5c0a6059034b7b880f1b3768064ce0` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:05.140335 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8122b3f6-dfad-48da-bf70-f6644538707f admin admin] RBAC: Authorizing `identity:delete_project(project_id=6a5c0a6059034b7b880f1b3768064ce0)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:28:05.147869 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8122b3f6-dfad-48da-bf70-f6644538707f admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:28:05.252364 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 855/1716] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 09:28:05 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:28:05.254034 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2727f329-05b5-4fab-b0f3-eeafde6afe92 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:28:05.260145 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-2727f329-05b5-4fab-b0f3-eeafde6afe92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:28:05.290842 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2727f329-05b5-4fab-b0f3-eeafde6afe92 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:28:05.331939 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-8122b3f6-dfad-48da-bf70-f6644538707f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6a5c0a6059034b7b880f1b3768064ce0', 'request_id': 'req-8122b3f6-dfad-48da-bf70-f6644538707f'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:28:05.340959 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-8122b3f6-dfad-48da-bf70-f6644538707f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6a5c0a6059034b7b880f1b3768064ce0', 'request_id': 'req-8122b3f6-dfad-48da-bf70-f6644538707f'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:28:05.383089 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:28:05.393831 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-8122b3f6-dfad-48da-bf70-f6644538707f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6a5c0a6059034b7b880f1b3768064ce0', 'request_id': 'req-8122b3f6-dfad-48da-bf70-f6644538707f'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:28:05.425874 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b3b0ac-41d4-4c5a-9c40-26f6a674aac7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c32b83577c46488906fe409a790b24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:28:05.426517 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b3b0ac-41d4-4c5a-9c40-26f6a674aac7 service cinder] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:05.427023 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b3b0ac-41d4-4c5a-9c40-26f6a674aac7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:05.427542 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b3b0ac-41d4-4c5a-9c40-26f6a674aac7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:05.429660 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-86b3b0ac-41d4-4c5a-9c40-26f6a674aac7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:28:05.503859 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-8122b3f6-dfad-48da-bf70-f6644538707f admin admin] The token cache is being invalidate because project 6a5c0a6059034b7b880f1b3768064ce0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:28:05.504757 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-8122b3f6-dfad-48da-bf70-f6644538707f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:28:05.505924 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-8122b3f6-dfad-48da-bf70-f6644538707f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:28:05.507008 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 862/1717] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 24 09:28:04 2022] DELETE /identity/v3/projects/6a5c0a6059034b7b880f1b3768064ce0 => generated 0 bytes in 554 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:28:05.512376 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8122b3f6-dfad-48da-bf70-f6644538707f admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:28:05.515173 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-8122b3f6-dfad-48da-bf70-f6644538707f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:28:05.572074 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b3b0ac-41d4-4c5a-9c40-26f6a674aac7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:28:05.575053 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b3b0ac-41d4-4c5a-9c40-26f6a674aac7 service cinder] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:28:05.624194 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 856/1718] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:28:05 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:28:05.625320 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b3b0ac-41d4-4c5a-9c40-26f6a674aac7 service cinder] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:28:05.645700 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b3b0ac-41d4-4c5a-9c40-26f6a674aac7 service cinder] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:28:05.652721 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897bed14-b85c-4cc8-8abd-da8c28b364c0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c32b83577c46488906fe409a790b24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:28:05.653436 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897bed14-b85c-4cc8-8abd-da8c28b364c0 service cinder] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:05.653786 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897bed14-b85c-4cc8-8abd-da8c28b364c0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:05.654222 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897bed14-b85c-4cc8-8abd-da8c28b364c0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:05.665251 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-897bed14-b85c-4cc8-8abd-da8c28b364c0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:28:05.668501 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8122b3f6-dfad-48da-bf70-f6644538707f admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:28:05.681247 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95981eca-5ece-4410-9bfe-0f266ede6ec4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:28:05.681910 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95981eca-5ece-4410-9bfe-0f266ede6ec4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:05.690083 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95981eca-5ece-4410-9bfe-0f266ede6ec4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:05.690525 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95981eca-5ece-4410-9bfe-0f266ede6ec4 admin admin] PATH_INFO: `/v3/users/1a142d6b2a774637a141aa10d15802c8` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:05.693099 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95981eca-5ece-4410-9bfe-0f266ede6ec4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1a142d6b2a774637a141aa10d15802c8)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:28:05.696149 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95981eca-5ece-4410-9bfe-0f266ede6ec4 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:28:05.812332 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897bed14-b85c-4cc8-8abd-da8c28b364c0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:28:05.814753 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897bed14-b85c-4cc8-8abd-da8c28b364c0 service cinder] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:28:05.836303 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 857/1719] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:28:05 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:28:05.945471 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-95981eca-5ece-4410-9bfe-0f266ede6ec4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1a142d6b2a774637a141aa10d15802c8'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:28:05.949417 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-95981eca-5ece-4410-9bfe-0f266ede6ec4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1a142d6b2a774637a141aa10d15802c8'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:28:05.975930 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-95981eca-5ece-4410-9bfe-0f266ede6ec4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1a142d6b2a774637a141aa10d15802c8'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:28:06.011413 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:28:06.020553 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-95981eca-5ece-4410-9bfe-0f266ede6ec4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1a142d6b2a774637a141aa10d15802c8'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:28:06.076915 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:28:06.096511 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-95981eca-5ece-4410-9bfe-0f266ede6ec4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1a142d6b2a774637a141aa10d15802c8'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:28:06.103827 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 863/1720] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 24 09:28:05 2022] DELETE /identity/v3/users/1a142d6b2a774637a141aa10d15802c8 => generated 0 bytes in 592 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:28:06.298983 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897bed14-b85c-4cc8-8abd-da8c28b364c0 service cinder] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:28:06.303675 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-897bed14-b85c-4cc8-8abd-da8c28b364c0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:28:06.328253 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897bed14-b85c-4cc8-8abd-da8c28b364c0 service cinder] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:28:06.519052 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39f0c4f-ca1d-4bc6-b163-395967aa54a0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c32b83577c46488906fe409a790b24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:28:06.519052 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39f0c4f-ca1d-4bc6-b163-395967aa54a0 service cinder] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:06.519052 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39f0c4f-ca1d-4bc6-b163-395967aa54a0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:06.519052 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39f0c4f-ca1d-4bc6-b163-395967aa54a0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:06.530823 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-b39f0c4f-ca1d-4bc6-b163-395967aa54a0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:28:06.672145 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39f0c4f-ca1d-4bc6-b163-395967aa54a0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:28:06.673605 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39f0c4f-ca1d-4bc6-b163-395967aa54a0 service cinder] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:28:06.704213 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 858/1721] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:28:06 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:28:06.894523 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9931abc-ed49-4abe-95be-45a0879b5621 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:06.895053 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9931abc-ed49-4abe-95be-45a0879b5621 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:06.895125 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9931abc-ed49-4abe-95be-45a0879b5621 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:06.953534 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa53bcca-946e-4d58-9090-0d01fac36e41 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:06.953831 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa53bcca-946e-4d58-9090-0d01fac36e41 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:06.954106 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa53bcca-946e-4d58-9090-0d01fac36e41 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:06.955851 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 859/1722] 10.222.0.16 () {54 vars in 912 bytes} [Wed Aug 24 09:28:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:28:06.974368 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7176c33-bc2a-4ef2-aea6-4d4268454ab2 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:06.975276 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7176c33-bc2a-4ef2-aea6-4d4268454ab2 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:06.975794 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7176c33-bc2a-4ef2-aea6-4d4268454ab2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:06.990363 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-a9931abc-ed49-4abe-95be-45a0879b5621 None None] MFA Rules not processed for user `cbcaa84bc9c54d7aa4178d0174a5936e`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:28:06.991745 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-a9931abc-ed49-4abe-95be-45a0879b5621 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:28:07.083569 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-b7176c33-bc2a-4ef2-aea6-4d4268454ab2 None None] MFA Rules not processed for user `8e5260e798404e8bb12b25ca8ea97052`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:28:07.085560 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-b7176c33-bc2a-4ef2-aea6-4d4268454ab2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:28:07.255033 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 864/1723] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 09:28:06 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 361 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:28:07.271264 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9931abc-ed49-4abe-95be-45a0879b5621 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:28:07.274057 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-a9931abc-ed49-4abe-95be-45a0879b5621 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:28:07.309803 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 860/1724] 10.222.0.16 () {58 vars in 1009 bytes} [Wed Aug 24 09:28:06 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 338 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:28:07.397549 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7176c33-bc2a-4ef2-aea6-4d4268454ab2 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:28:07.397549 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-b7176c33-bc2a-4ef2-aea6-4d4268454ab2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:28:07.404485 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7176c33-bc2a-4ef2-aea6-4d4268454ab2 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:28:07.423467 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a87141-1b36-4b8c-b700-40d092b450ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e5260e798404e8bb12b25ca8ea97052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:28:07.427664 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a87141-1b36-4b8c-b700-40d092b450ec service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:07.429120 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a87141-1b36-4b8c-b700-40d092b450ec service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:07.434895 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a87141-1b36-4b8c-b700-40d092b450ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:07.448428 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9931abc-ed49-4abe-95be-45a0879b5621 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:28:07.456276 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a87141-1b36-4b8c-b700-40d092b450ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:28:07.458832 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a87141-1b36-4b8c-b700-40d092b450ec service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:28:07.491012 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6442661-07a7-4732-9e6c-eefe2c8eef47 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f055f7dc51d43bda9b315ca67f11d58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:28:07.495531 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6442661-07a7-4732-9e6c-eefe2c8eef47 service nova] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:07.496063 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6442661-07a7-4732-9e6c-eefe2c8eef47 service nova] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:07.498874 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6442661-07a7-4732-9e6c-eefe2c8eef47 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:07.510228 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 861/1725] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:28:07 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 09:28:07.533227 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6442661-07a7-4732-9e6c-eefe2c8eef47 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:28:07.538669 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6442661-07a7-4732-9e6c-eefe2c8eef47 service nova] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:28:07.548324 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 865/1726] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:28:07 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 278 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:28:08.032170 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6442661-07a7-4732-9e6c-eefe2c8eef47 service nova] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:28:08.033236 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be8c56d-5ca7-41de-8bee-927f9f9ea132 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:08.033713 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be8c56d-5ca7-41de-8bee-927f9f9ea132 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:08.034118 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be8c56d-5ca7-41de-8bee-927f9f9ea132 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:08.036145 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-c6442661-07a7-4732-9e6c-eefe2c8eef47 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:28:08.084094 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-9be8c56d-5ca7-41de-8bee-927f9f9ea132 None None] MFA Rules not processed for user `def04335e57e48409a6943fa231d47c3`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:28:08.088849 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-9be8c56d-5ca7-41de-8bee-927f9f9ea132 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:28:08.158959 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:28:08.190505 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:28:08.198236 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6442661-07a7-4732-9e6c-eefe2c8eef47 service nova] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:28:08.205317 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab0d22d-0552-4c3d-9637-f4bcd10f9819 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:28:08.206280 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab0d22d-0552-4c3d-9637-f4bcd10f9819 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:08.206826 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab0d22d-0552-4c3d-9637-f4bcd10f9819 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:08.207352 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab0d22d-0552-4c3d-9637-f4bcd10f9819 admin admin] PATH_INFO: `/v3/projects/33e5b4a408f1463a848113782742f8cd` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:08.208849 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab0d22d-0552-4c3d-9637-f4bcd10f9819 admin admin] RBAC: Authorizing `identity:delete_project(project_id=33e5b4a408f1463a848113782742f8cd)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:28:08.212976 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab0d22d-0552-4c3d-9637-f4bcd10f9819 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:28:08.291638 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 862/1727] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 09:28:08 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:28:08.366476 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-bab0d22d-0552-4c3d-9637-f4bcd10f9819 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '33e5b4a408f1463a848113782742f8cd', 'request_id': 'req-bab0d22d-0552-4c3d-9637-f4bcd10f9819'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:28:08.383730 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-bab0d22d-0552-4c3d-9637-f4bcd10f9819 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '33e5b4a408f1463a848113782742f8cd', 'request_id': 'req-bab0d22d-0552-4c3d-9637-f4bcd10f9819'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:28:08.406818 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-bab0d22d-0552-4c3d-9637-f4bcd10f9819 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '33e5b4a408f1463a848113782742f8cd', 'request_id': 'req-bab0d22d-0552-4c3d-9637-f4bcd10f9819'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:28:08.535877 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-bab0d22d-0552-4c3d-9637-f4bcd10f9819 admin admin] The token cache is being invalidate because project 33e5b4a408f1463a848113782742f8cd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:28:08.537411 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-bab0d22d-0552-4c3d-9637-f4bcd10f9819 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:28:08.538291 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-bab0d22d-0552-4c3d-9637-f4bcd10f9819 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:28:08.541048 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 866/1728] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 24 09:28:08 2022] DELETE /identity/v3/projects/33e5b4a408f1463a848113782742f8cd => generated 0 bytes in 509 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:28:08.546196 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc8431e-9865-4143-afc0-6d3c3006fca8 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:08.547639 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc8431e-9865-4143-afc0-6d3c3006fca8 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:08.547930 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc8431e-9865-4143-afc0-6d3c3006fca8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:08.635497 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-edc8431e-9865-4143-afc0-6d3c3006fca8 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:28:08.637473 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-edc8431e-9865-4143-afc0-6d3c3006fca8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:28:08.825171 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe015cf-e14b-4116-8184-eaddcdc7fbed None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:08.825171 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe015cf-e14b-4116-8184-eaddcdc7fbed None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:08.825171 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe015cf-e14b-4116-8184-eaddcdc7fbed None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:08.826906 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 867/1729] 10.222.0.16 () {54 vars in 961 bytes} [Wed Aug 24 09:28:08 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:28:08.841373 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37893c75-a386-4a3c-af55-7cf324312e3f None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:08.842246 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37893c75-a386-4a3c-af55-7cf324312e3f None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:08.842620 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37893c75-a386-4a3c-af55-7cf324312e3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:08.893433 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-37893c75-a386-4a3c-af55-7cf324312e3f None None] MFA Rules not processed for user `80cada22039d4f11935412ee578acd05`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:28:08.894680 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-37893c75-a386-4a3c-af55-7cf324312e3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:28:08.952056 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 863/1730] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 09:28:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 407 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:28:08.962914 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321cdd67-33fe-4269-811e-0232ad7cfbbf None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:08.963387 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321cdd67-33fe-4269-811e-0232ad7cfbbf None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:08.964117 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321cdd67-33fe-4269-811e-0232ad7cfbbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:09.023991 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-321cdd67-33fe-4269-811e-0232ad7cfbbf None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:28:09.025881 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-321cdd67-33fe-4269-811e-0232ad7cfbbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:28:09.040047 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 864/1731] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 09:28:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:28:09.045258 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321cdd67-33fe-4269-811e-0232ad7cfbbf None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:28:09.073987 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 868/1732] 10.222.0.16 () {58 vars in 1058 bytes} [Wed Aug 24 09:28:08 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Aug 24 09:28:09.091051 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37893c75-a386-4a3c-af55-7cf324312e3f None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:28:09.096014 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321cdd67-33fe-4269-811e-0232ad7cfbbf None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:28:09.101855 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ae63bb-78a9-44a3-9a68-8cf90d6d0993 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:28:09.102913 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ae63bb-78a9-44a3-9a68-8cf90d6d0993 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:09.103443 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ae63bb-78a9-44a3-9a68-8cf90d6d0993 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:09.103961 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ae63bb-78a9-44a3-9a68-8cf90d6d0993 admin admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:09.105465 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ae63bb-78a9-44a3-9a68-8cf90d6d0993 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:28:09.105935 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ae63bb-78a9-44a3-9a68-8cf90d6d0993 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:28:09.108758 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ae63bb-78a9-44a3-9a68-8cf90d6d0993 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:28:09.121100 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37893c75-a386-4a3c-af55-7cf324312e3f None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:28:09.128888 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 865/1733] 10.222.0.16 () {58 vars in 1153 bytes} [Wed Aug 24 09:28:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:28:09.137367 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ae63bb-78a9-44a3-9a68-8cf90d6d0993 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:28:09.141278 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2fcdf7-0dcb-4422-a373-e14e5fa00468 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80cada22039d4f11935412ee578acd05', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:28:09.143430 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2fcdf7-0dcb-4422-a373-e14e5fa00468 service placement] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:09.144469 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2fcdf7-0dcb-4422-a373-e14e5fa00468 service placement] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:09.144849 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2fcdf7-0dcb-4422-a373-e14e5fa00468 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:09.150379 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-1e2fcdf7-0dcb-4422-a373-e14e5fa00468 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:28:09.164147 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ae63bb-78a9-44a3-9a68-8cf90d6d0993 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:28:09.174792 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ce0e73-65a2-407b-8982-d1befe419766 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:28:09.175828 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ce0e73-65a2-407b-8982-d1befe419766 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:09.176809 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2fcdf7-0dcb-4422-a373-e14e5fa00468 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:28:09.177184 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ce0e73-65a2-407b-8982-d1befe419766 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:09.178943 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2fcdf7-0dcb-4422-a373-e14e5fa00468 service placement] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:28:09.179371 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ce0e73-65a2-407b-8982-d1befe419766 admin admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:09.180258 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ce0e73-65a2-407b-8982-d1befe419766 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:28:09.182774 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ce0e73-65a2-407b-8982-d1befe419766 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:28:09.221534 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 869/1734] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:28:09 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:28:09.340810 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e61eb2-fa8e-476f-bce0-c40f2a110cf7 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:09.344836 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e61eb2-fa8e-476f-bce0-c40f2a110cf7 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:09.344985 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 866/1735] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 24 09:28:09 2022] POST /identity/v3/projects => generated 377 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:28:09.345087 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e61eb2-fa8e-476f-bce0-c40f2a110cf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:09.346245 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ce0e73-65a2-407b-8982-d1befe419766 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:28:09.403431 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-57e61eb2-fa8e-476f-bce0-c40f2a110cf7 None None] MFA Rules not processed for user `b9099dc3ad92431cbac98f144690038c`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:28:09.411411 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-57e61eb2-fa8e-476f-bce0-c40f2a110cf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:28:09.439611 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ce0e73-65a2-407b-8982-d1befe419766 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:28:09.446912 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55086b5-ebb7-480f-8db2-cf600ad76eb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:28:09.448565 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55086b5-ebb7-480f-8db2-cf600ad76eb3 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:09.448565 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55086b5-ebb7-480f-8db2-cf600ad76eb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:09.448761 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55086b5-ebb7-480f-8db2-cf600ad76eb3 admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:09.449391 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55086b5-ebb7-480f-8db2-cf600ad76eb3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:28:09.452311 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55086b5-ebb7-480f-8db2-cf600ad76eb3 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:28:09.534999 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 867/1736] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 24 09:28:09 2022] POST /identity/v3/users => generated 289 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:28:09.548572 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55086b5-ebb7-480f-8db2-cf600ad76eb3 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:28:09.557641 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55086b5-ebb7-480f-8db2-cf600ad76eb3 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:28:09.570081 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bdadf2-8d81-4388-909b-276cce81943f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:28:09.570327 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 870/1737] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 09:28:09 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 09:28:09.571204 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bdadf2-8d81-4388-909b-276cce81943f admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:09.571507 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bdadf2-8d81-4388-909b-276cce81943f admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:09.577144 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bdadf2-8d81-4388-909b-276cce81943f admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:09.577144 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bdadf2-8d81-4388-909b-276cce81943f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:28:09.577144 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bdadf2-8d81-4388-909b-276cce81943f admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:28:09.604052 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e61eb2-fa8e-476f-bce0-c40f2a110cf7 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:28:09.604052 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-57e61eb2-fa8e-476f-bce0-c40f2a110cf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:28:09.626180 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 868/1738] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 24 09:28:09 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:28:09.628744 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290c0415-c57a-4adc-9483-b78bcc916dc0 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:09.629322 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290c0415-c57a-4adc-9483-b78bcc916dc0 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:09.630803 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290c0415-c57a-4adc-9483-b78bcc916dc0 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:09.633856 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 869/1739] 10.222.0.16 () {54 vars in 912 bytes} [Wed Aug 24 09:28:09 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:28:09.640402 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290c0415-c57a-4adc-9483-b78bcc916dc0 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:28:09.650793 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:28:09.664613 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290c0415-c57a-4adc-9483-b78bcc916dc0 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:28:09.676196 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b939191-8a1c-49ee-a9fb-9e9dc621d929 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:28:09.679323 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b939191-8a1c-49ee-a9fb-9e9dc621d929 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:09.679941 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b939191-8a1c-49ee-a9fb-9e9dc621d929 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:09.679941 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b939191-8a1c-49ee-a9fb-9e9dc621d929 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:09.680691 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b939191-8a1c-49ee-a9fb-9e9dc621d929 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:28:09.682683 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b939191-8a1c-49ee-a9fb-9e9dc621d929 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:28:09.697516 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 870/1740] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 24 09:28:09 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:28:09.698640 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c831104b-76d3-471a-9881-3f3c82338d2a None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:09.698923 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c831104b-76d3-471a-9881-3f3c82338d2a None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:09.699197 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c831104b-76d3-471a-9881-3f3c82338d2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:09.714546 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:28:09.726519 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-c831104b-76d3-471a-9881-3f3c82338d2a None None] MFA Rules not processed for user `8e5260e798404e8bb12b25ca8ea97052`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:28:09.726519 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-c831104b-76d3-471a-9881-3f3c82338d2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:28:09.758144 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e61eb2-fa8e-476f-bce0-c40f2a110cf7 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:28:09.764701 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15856708-41aa-40a6-84e6-fe33faaecb49 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f055f7dc51d43bda9b315ca67f11d58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:28:09.765329 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15856708-41aa-40a6-84e6-fe33faaecb49 service nova] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:09.765677 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15856708-41aa-40a6-84e6-fe33faaecb49 service nova] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:09.765951 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15856708-41aa-40a6-84e6-fe33faaecb49 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:09.783665 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15856708-41aa-40a6-84e6-fe33faaecb49 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:28:09.785777 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15856708-41aa-40a6-84e6-fe33faaecb49 service nova] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:28:09.807933 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 871/1741] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:28:09 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:28:09.813804 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15856708-41aa-40a6-84e6-fe33faaecb49 service nova] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:28:09.855705 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 871/1742] 10.222.0.16 () {58 vars in 1009 bytes} [Wed Aug 24 09:28:09 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:28:09.876528 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c831104b-76d3-471a-9881-3f3c82338d2a None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:28:09.879850 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-c831104b-76d3-471a-9881-3f3c82338d2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:28:09.881514 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15856708-41aa-40a6-84e6-fe33faaecb49 service nova] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:28:09.895296 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c831104b-76d3-471a-9881-3f3c82338d2a None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:28:09.901088 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14081c1f-03cc-4c4b-829b-e9559ad4cddb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e5260e798404e8bb12b25ca8ea97052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:28:09.902052 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb88f9ec-1291-4ec8-a051-039d26eb7484 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:28:09.902503 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14081c1f-03cc-4c4b-829b-e9559ad4cddb service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:09.902887 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14081c1f-03cc-4c4b-829b-e9559ad4cddb service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:09.903564 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14081c1f-03cc-4c4b-829b-e9559ad4cddb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:09.904835 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb88f9ec-1291-4ec8-a051-039d26eb7484 admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:09.908268 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb88f9ec-1291-4ec8-a051-039d26eb7484 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:09.908797 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb88f9ec-1291-4ec8-a051-039d26eb7484 admin admin] PATH_INFO: `/v3/projects/60909f2642b94bdb8fae1b3621f1e2df/users/87c2b04bd41143d89585d8df8dcf7873/roles/136cdf78e00a426997dd4e37faff489b` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:09.995069 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14081c1f-03cc-4c4b-829b-e9559ad4cddb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:28:10.002600 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14081c1f-03cc-4c4b-829b-e9559ad4cddb service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:28:10.004375 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb88f9ec-1291-4ec8-a051-039d26eb7484 admin admin] RBAC: Authorizing `identity:create_grant(project_id=60909f2642b94bdb8fae1b3621f1e2df, user_id=87c2b04bd41143d89585d8df8dcf7873, role_id=136cdf78e00a426997dd4e37faff489b)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:28:10.015103 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb88f9ec-1291-4ec8-a051-039d26eb7484 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:28:10.034417 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 872/1743] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:28:09 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:28:10.098639 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 872/1744] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 24 09:28:09 2022] PUT /identity/v3/projects/60909f2642b94bdb8fae1b3621f1e2df/users/87c2b04bd41143d89585d8df8dcf7873/roles/136cdf78e00a426997dd4e37faff489b => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:28:10.111195 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bed655a-1ed5-4ba9-9b80-a4e309b6cddc None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:10.111727 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bed655a-1ed5-4ba9-9b80-a4e309b6cddc None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:10.111822 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bed655a-1ed5-4ba9-9b80-a4e309b6cddc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:10.183892 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-1bed655a-1ed5-4ba9-9b80-a4e309b6cddc None None] MFA Rules not processed for user `87c2b04bd41143d89585d8df8dcf7873`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:28:10.187928 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-1bed655a-1ed5-4ba9-9b80-a4e309b6cddc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:28:10.404827 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 873/1745] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 09:28:10 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 294 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:28:10.409444 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb88f9ec-1291-4ec8-a051-039d26eb7484 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:28:10.558058 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb88f9ec-1291-4ec8-a051-039d26eb7484 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:28:10.588723 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2918ef89-240f-4fbc-8ab9-fc2414358186 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:28:10.589036 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2918ef89-240f-4fbc-8ab9-fc2414358186 admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:10.589323 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2918ef89-240f-4fbc-8ab9-fc2414358186 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:10.589601 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2918ef89-240f-4fbc-8ab9-fc2414358186 admin admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:10.592949 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2918ef89-240f-4fbc-8ab9-fc2414358186 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:28:10.594943 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2918ef89-240f-4fbc-8ab9-fc2414358186 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:28:10.738557 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bed655a-1ed5-4ba9-9b80-a4e309b6cddc None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:28:10.747501 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 873/1746] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 24 09:28:10 2022] POST /identity/v3/projects => generated 381 bytes in 337 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:28:10.752788 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bed655a-1ed5-4ba9-9b80-a4e309b6cddc None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:28:10.889652 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cceb15a3-02f1-40fb-bda0-12ca8f341ce3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:28:10.890285 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cceb15a3-02f1-40fb-bda0-12ca8f341ce3 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:10.890633 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cceb15a3-02f1-40fb-bda0-12ca8f341ce3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:10.890975 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cceb15a3-02f1-40fb-bda0-12ca8f341ce3 admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:10.891730 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cceb15a3-02f1-40fb-bda0-12ca8f341ce3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:28:10.894336 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cceb15a3-02f1-40fb-bda0-12ca8f341ce3 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:28:10.994938 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 874/1747] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 24 09:28:10 2022] POST /identity/v3/users => generated 297 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 09:28:11.001380 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2918ef89-240f-4fbc-8ab9-fc2414358186 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:28:11.027486 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2918ef89-240f-4fbc-8ab9-fc2414358186 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:28:11.044207 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de76f13f-20ad-454d-8cf6-8f6b635cac86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:28:11.047665 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de76f13f-20ad-454d-8cf6-8f6b635cac86 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:11.048273 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de76f13f-20ad-454d-8cf6-8f6b635cac86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:11.052143 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de76f13f-20ad-454d-8cf6-8f6b635cac86 admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:11.056308 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de76f13f-20ad-454d-8cf6-8f6b635cac86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:28:11.065031 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de76f13f-20ad-454d-8cf6-8f6b635cac86 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:28:11.081730 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 874/1748] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 24 09:28:10 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:28:11.084580 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cceb15a3-02f1-40fb-bda0-12ca8f341ce3 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:28:11.124370 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cceb15a3-02f1-40fb-bda0-12ca8f341ce3 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:28:11.137162 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5594be-54b7-4ba0-be32-733da349404e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:28:11.142561 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5594be-54b7-4ba0-be32-733da349404e admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:11.145201 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5594be-54b7-4ba0-be32-733da349404e admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:11.151380 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5594be-54b7-4ba0-be32-733da349404e admin admin] PATH_INFO: `/v3/projects/b9c3da8e76534451952040e090603d0b/users/caec843f49fa491db2b8c5565794940c/roles/15f634d4a2d247ddab90c458e5e1c141` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:11.166315 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:28:11.197078 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5594be-54b7-4ba0-be32-733da349404e admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9c3da8e76534451952040e090603d0b, user_id=caec843f49fa491db2b8c5565794940c, role_id=15f634d4a2d247ddab90c458e5e1c141)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:28:11.200245 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5594be-54b7-4ba0-be32-733da349404e admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:28:11.309841 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 875/1749] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 24 09:28:11 2022] PUT /identity/v3/projects/b9c3da8e76534451952040e090603d0b/users/caec843f49fa491db2b8c5565794940c/roles/15f634d4a2d247ddab90c458e5e1c141 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:28:11.315323 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de76f13f-20ad-454d-8cf6-8f6b635cac86 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:28:11.349721 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de76f13f-20ad-454d-8cf6-8f6b635cac86 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:28:11.371528 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:28:11.534772 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47aa1b9c-262e-45af-a538-c509b4f58ba6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:28:11.535195 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47aa1b9c-262e-45af-a538-c509b4f58ba6 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:11.535557 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47aa1b9c-262e-45af-a538-c509b4f58ba6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:11.535860 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47aa1b9c-262e-45af-a538-c509b4f58ba6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:11.536467 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47aa1b9c-262e-45af-a538-c509b4f58ba6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:28:11.538093 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47aa1b9c-262e-45af-a538-c509b4f58ba6 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:28:11.578963 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 875/1750] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 24 09:28:11 2022] GET /identity/v3/roles => generated 1420 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:28:11.580035 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5594be-54b7-4ba0-be32-733da349404e admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:28:11.606809 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5594be-54b7-4ba0-be32-733da349404e admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:28:11.618405 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70093181-26d8-4804-9ddd-e1a1364f2ab5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:28:11.620767 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70093181-26d8-4804-9ddd-e1a1364f2ab5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:11.626098 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70093181-26d8-4804-9ddd-e1a1364f2ab5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:11.628187 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70093181-26d8-4804-9ddd-e1a1364f2ab5 admin admin] PATH_INFO: `/v3/projects/b9c3da8e76534451952040e090603d0b/users/caec843f49fa491db2b8c5565794940c/roles/15f634d4a2d247ddab90c458e5e1c141` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:11.631442 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70093181-26d8-4804-9ddd-e1a1364f2ab5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9c3da8e76534451952040e090603d0b, user_id=caec843f49fa491db2b8c5565794940c, role_id=15f634d4a2d247ddab90c458e5e1c141)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:28:11.639246 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70093181-26d8-4804-9ddd-e1a1364f2ab5 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:28:11.691968 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 876/1751] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 24 09:28:11 2022] PUT /identity/v3/projects/b9c3da8e76534451952040e090603d0b/users/caec843f49fa491db2b8c5565794940c/roles/15f634d4a2d247ddab90c458e5e1c141 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:28:11.692387 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59475c2f-7516-487f-8728-273053eb0257 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:11.692387 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59475c2f-7516-487f-8728-273053eb0257 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:11.692387 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59475c2f-7516-487f-8728-273053eb0257 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:11.778570 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-59475c2f-7516-487f-8728-273053eb0257 None None] MFA Rules not processed for user `caec843f49fa491db2b8c5565794940c`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:28:11.780734 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-59475c2f-7516-487f-8728-273053eb0257 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:28:11.967708 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 876/1752] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 09:28:11 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:28:12.007798 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70093181-26d8-4804-9ddd-e1a1364f2ab5 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:28:12.008229 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-70093181-26d8-4804-9ddd-e1a1364f2ab5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:28:12.103707 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70093181-26d8-4804-9ddd-e1a1364f2ab5 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:28:12.107753 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b3ef5b-7cec-4aba-a4c7-3b3eb8f3c982 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c32b83577c46488906fe409a790b24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:28:12.108386 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b3ef5b-7cec-4aba-a4c7-3b3eb8f3c982 service cinder] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:12.108662 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b3ef5b-7cec-4aba-a4c7-3b3eb8f3c982 service cinder] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:12.108981 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b3ef5b-7cec-4aba-a4c7-3b3eb8f3c982 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:12.128709 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b3ef5b-7cec-4aba-a4c7-3b3eb8f3c982 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:28:12.130435 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b3ef5b-7cec-4aba-a4c7-3b3eb8f3c982 service cinder] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:28:12.159994 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 877/1753] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:28:12 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:28:13.123973 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db15211a-05ab-4159-9a9a-23a5a3fd9132 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:13.124926 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db15211a-05ab-4159-9a9a-23a5a3fd9132 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:13.125356 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db15211a-05ab-4159-9a9a-23a5a3fd9132 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:13.169282 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-db15211a-05ab-4159-9a9a-23a5a3fd9132 None None] MFA Rules not processed for user `cbcaa84bc9c54d7aa4178d0174a5936e`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:28:13.170798 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-db15211a-05ab-4159-9a9a-23a5a3fd9132 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:28:13.280011 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 877/1754] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 09:28:13 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:28:13.784409 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b3ef5b-7cec-4aba-a4c7-3b3eb8f3c982 service cinder] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:28:13.808028 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b3ef5b-7cec-4aba-a4c7-3b3eb8f3c982 service cinder] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:28:13.820036 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0713951-2e76-41e0-9eb7-1333a8a7031a tempest-VolumeQuotaClassesTest-1940638631 tempest-VolumeQuotaClassesTest-1940638631-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'caec843f49fa491db2b8c5565794940c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9c3da8e76534451952040e090603d0b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:28:13.820819 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0713951-2e76-41e0-9eb7-1333a8a7031a tempest-VolumeQuotaClassesTest-1940638631 tempest-VolumeQuotaClassesTest-1940638631-project-admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:13.821173 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0713951-2e76-41e0-9eb7-1333a8a7031a tempest-VolumeQuotaClassesTest-1940638631 tempest-VolumeQuotaClassesTest-1940638631-project-admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:13.821520 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0713951-2e76-41e0-9eb7-1333a8a7031a tempest-VolumeQuotaClassesTest-1940638631 tempest-VolumeQuotaClassesTest-1940638631-project-admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:13.822304 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0713951-2e76-41e0-9eb7-1333a8a7031a tempest-VolumeQuotaClassesTest-1940638631 tempest-VolumeQuotaClassesTest-1940638631-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:28:13.822794 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0713951-2e76-41e0-9eb7-1333a8a7031a tempest-VolumeQuotaClassesTest-1940638631 tempest-VolumeQuotaClassesTest-1940638631-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:28:13.825026 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0713951-2e76-41e0-9eb7-1333a8a7031a tempest-VolumeQuotaClassesTest-1940638631 tempest-VolumeQuotaClassesTest-1940638631-project-admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:28:13.841546 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 878/1755] 10.222.0.16 () {58 vars in 1153 bytes} [Wed Aug 24 09:28:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:28:13.845573 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db15211a-05ab-4159-9a9a-23a5a3fd9132 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:28:13.865776 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db15211a-05ab-4159-9a9a-23a5a3fd9132 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:28:13.876540 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246845b1-e580-42e8-bc3a-e82cfe563f6b tempest-VolumeQuotaClassesTest-1940638631 tempest-VolumeQuotaClassesTest-1940638631-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'caec843f49fa491db2b8c5565794940c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9c3da8e76534451952040e090603d0b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:28:13.877989 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246845b1-e580-42e8-bc3a-e82cfe563f6b tempest-VolumeQuotaClassesTest-1940638631 tempest-VolumeQuotaClassesTest-1940638631-project-admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:13.877989 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246845b1-e580-42e8-bc3a-e82cfe563f6b tempest-VolumeQuotaClassesTest-1940638631 tempest-VolumeQuotaClassesTest-1940638631-project-admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:13.879156 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246845b1-e580-42e8-bc3a-e82cfe563f6b tempest-VolumeQuotaClassesTest-1940638631 tempest-VolumeQuotaClassesTest-1940638631-project-admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:13.882642 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246845b1-e580-42e8-bc3a-e82cfe563f6b tempest-VolumeQuotaClassesTest-1940638631 tempest-VolumeQuotaClassesTest-1940638631-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:28:13.884688 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246845b1-e580-42e8-bc3a-e82cfe563f6b tempest-VolumeQuotaClassesTest-1940638631 tempest-VolumeQuotaClassesTest-1940638631-project-admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:28:14.024425 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 878/1756] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 24 09:28:13 2022] POST /identity/v3/projects => generated 353 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:28:14.029853 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0713951-2e76-41e0-9eb7-1333a8a7031a tempest-VolumeQuotaClassesTest-1940638631 tempest-VolumeQuotaClassesTest-1940638631-project-admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:28:14.045421 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0713951-2e76-41e0-9eb7-1333a8a7031a tempest-VolumeQuotaClassesTest-1940638631 tempest-VolumeQuotaClassesTest-1940638631-project-admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:28:14.139384 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4651cf5-4913-4390-91ef-926b91e04c88 tempest-VolumeQuotaClassesTest-1940638631 tempest-VolumeQuotaClassesTest-1940638631-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'caec843f49fa491db2b8c5565794940c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9c3da8e76534451952040e090603d0b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:28:14.140151 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4651cf5-4913-4390-91ef-926b91e04c88 tempest-VolumeQuotaClassesTest-1940638631 tempest-VolumeQuotaClassesTest-1940638631-project-admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:14.140547 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4651cf5-4913-4390-91ef-926b91e04c88 tempest-VolumeQuotaClassesTest-1940638631 tempest-VolumeQuotaClassesTest-1940638631-project-admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:14.140891 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4651cf5-4913-4390-91ef-926b91e04c88 tempest-VolumeQuotaClassesTest-1940638631 tempest-VolumeQuotaClassesTest-1940638631-project-admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:14.141683 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4651cf5-4913-4390-91ef-926b91e04c88 tempest-VolumeQuotaClassesTest-1940638631 tempest-VolumeQuotaClassesTest-1940638631-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:28:14.142138 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4651cf5-4913-4390-91ef-926b91e04c88 tempest-VolumeQuotaClassesTest-1940638631 tempest-VolumeQuotaClassesTest-1940638631-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:28:14.144857 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4651cf5-4913-4390-91ef-926b91e04c88 tempest-VolumeQuotaClassesTest-1940638631 tempest-VolumeQuotaClassesTest-1940638631-project-admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:28:14.183993 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 879/1757] 10.222.0.16 () {58 vars in 1153 bytes} [Wed Aug 24 09:28:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:28:14.281891 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246845b1-e580-42e8-bc3a-e82cfe563f6b tempest-VolumeQuotaClassesTest-1940638631 tempest-VolumeQuotaClassesTest-1940638631-project-admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:28:14.293575 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246845b1-e580-42e8-bc3a-e82cfe563f6b tempest-VolumeQuotaClassesTest-1940638631 tempest-VolumeQuotaClassesTest-1940638631-project-admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:28:14.318221 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0950e0e6-6bd6-4be6-aec8-0ee2ea045e99 tempest-VolumeQuotaClassesTest-1940638631 tempest-VolumeQuotaClassesTest-1940638631-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'caec843f49fa491db2b8c5565794940c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9c3da8e76534451952040e090603d0b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:28:14.318221 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0950e0e6-6bd6-4be6-aec8-0ee2ea045e99 tempest-VolumeQuotaClassesTest-1940638631 tempest-VolumeQuotaClassesTest-1940638631-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:14.318221 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0950e0e6-6bd6-4be6-aec8-0ee2ea045e99 tempest-VolumeQuotaClassesTest-1940638631 tempest-VolumeQuotaClassesTest-1940638631-project-admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:14.319085 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0950e0e6-6bd6-4be6-aec8-0ee2ea045e99 tempest-VolumeQuotaClassesTest-1940638631 tempest-VolumeQuotaClassesTest-1940638631-project-admin] PATH_INFO: `/v3/projects/ebf5af9017aa4f768aa12c830128b1c0` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:14.319085 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0950e0e6-6bd6-4be6-aec8-0ee2ea045e99 tempest-VolumeQuotaClassesTest-1940638631 tempest-VolumeQuotaClassesTest-1940638631-project-admin] RBAC: Authorizing `identity:delete_project(project_id=ebf5af9017aa4f768aa12c830128b1c0)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:28:14.319085 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0950e0e6-6bd6-4be6-aec8-0ee2ea045e99 tempest-VolumeQuotaClassesTest-1940638631 tempest-VolumeQuotaClassesTest-1940638631-project-admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:28:14.324694 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:28:14.363121 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54111568-fda7-4099-80a5-b32b8c58baba None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:14.363750 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54111568-fda7-4099-80a5-b32b8c58baba None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:14.364245 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54111568-fda7-4099-80a5-b32b8c58baba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:14.390794 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-54111568-fda7-4099-80a5-b32b8c58baba None None] MFA Rules not processed for user `b9099dc3ad92431cbac98f144690038c`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:28:14.392535 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-54111568-fda7-4099-80a5-b32b8c58baba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:28:14.486363 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-0950e0e6-6bd6-4be6-aec8-0ee2ea045e99 tempest-VolumeQuotaClassesTest-1940638631 tempest-VolumeQuotaClassesTest-1940638631-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ebf5af9017aa4f768aa12c830128b1c0', 'request_id': 'req-0950e0e6-6bd6-4be6-aec8-0ee2ea045e99'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:28:14.493272 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:28:14.501675 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-0950e0e6-6bd6-4be6-aec8-0ee2ea045e99 tempest-VolumeQuotaClassesTest-1940638631 tempest-VolumeQuotaClassesTest-1940638631-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ebf5af9017aa4f768aa12c830128b1c0', 'request_id': 'req-0950e0e6-6bd6-4be6-aec8-0ee2ea045e99'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:28:14.507456 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 880/1758] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 09:28:14 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 09:28:14.519719 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-0950e0e6-6bd6-4be6-aec8-0ee2ea045e99 tempest-VolumeQuotaClassesTest-1940638631 tempest-VolumeQuotaClassesTest-1940638631-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ebf5af9017aa4f768aa12c830128b1c0', 'request_id': 'req-0950e0e6-6bd6-4be6-aec8-0ee2ea045e99'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:28:14.606485 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-0950e0e6-6bd6-4be6-aec8-0ee2ea045e99 tempest-VolumeQuotaClassesTest-1940638631 tempest-VolumeQuotaClassesTest-1940638631-project-admin] The token cache is being invalidate because project ebf5af9017aa4f768aa12c830128b1c0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:28:14.606724 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-0950e0e6-6bd6-4be6-aec8-0ee2ea045e99 tempest-VolumeQuotaClassesTest-1940638631 tempest-VolumeQuotaClassesTest-1940638631-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:28:14.607749 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-0950e0e6-6bd6-4be6-aec8-0ee2ea045e99 tempest-VolumeQuotaClassesTest-1940638631 tempest-VolumeQuotaClassesTest-1940638631-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:28:14.609768 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 879/1759] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 24 09:28:14 2022] DELETE /identity/v3/projects/ebf5af9017aa4f768aa12c830128b1c0 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:28:15.545189 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54111568-fda7-4099-80a5-b32b8c58baba None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:28:15.671091 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54111568-fda7-4099-80a5-b32b8c58baba None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:28:15.742604 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1479f933-f3b0-4f4f-b465-e87dc4f3cd55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:28:15.743323 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1479f933-f3b0-4f4f-b465-e87dc4f3cd55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:15.743619 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1479f933-f3b0-4f4f-b465-e87dc4f3cd55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:15.743898 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1479f933-f3b0-4f4f-b465-e87dc4f3cd55 admin admin] PATH_INFO: `/v3/users/87c2b04bd41143d89585d8df8dcf7873` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:15.747870 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1479f933-f3b0-4f4f-b465-e87dc4f3cd55 admin admin] RBAC: Authorizing `identity:delete_user(user_id=87c2b04bd41143d89585d8df8dcf7873)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:28:15.754256 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1479f933-f3b0-4f4f-b465-e87dc4f3cd55 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:28:16.017499 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-1479f933-f3b0-4f4f-b465-e87dc4f3cd55 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '87c2b04bd41143d89585d8df8dcf7873'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:28:16.018320 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-1479f933-f3b0-4f4f-b465-e87dc4f3cd55 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '87c2b04bd41143d89585d8df8dcf7873'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:28:16.120845 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-1479f933-f3b0-4f4f-b465-e87dc4f3cd55 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '87c2b04bd41143d89585d8df8dcf7873'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:28:16.123381 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-1479f933-f3b0-4f4f-b465-e87dc4f3cd55 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '87c2b04bd41143d89585d8df8dcf7873'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:28:16.133071 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-1479f933-f3b0-4f4f-b465-e87dc4f3cd55 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '87c2b04bd41143d89585d8df8dcf7873'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:28:16.154706 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 881/1760] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 24 09:28:15 2022] DELETE /identity/v3/users/87c2b04bd41143d89585d8df8dcf7873 => generated 0 bytes in 610 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:28:16.159563 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfae36d-c97d-44e8-a051-dabb54d647f1 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:16.160057 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfae36d-c97d-44e8-a051-dabb54d647f1 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:16.160217 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfae36d-c97d-44e8-a051-dabb54d647f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:16.195479 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-7bfae36d-c97d-44e8-a051-dabb54d647f1 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:28:16.197006 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-7bfae36d-c97d-44e8-a051-dabb54d647f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:28:16.308283 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 880/1761] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 09:28:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:28:16.318936 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1479f933-f3b0-4f4f-b465-e87dc4f3cd55 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:28:16.320855 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-1479f933-f3b0-4f4f-b465-e87dc4f3cd55 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:28:16.361110 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:28:16.398096 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1479f933-f3b0-4f4f-b465-e87dc4f3cd55 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:28:16.402129 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c10ef4e-5871-43ee-8c2c-c3029bbeb5c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e5260e798404e8bb12b25ca8ea97052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:28:16.402759 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c10ef4e-5871-43ee-8c2c-c3029bbeb5c2 service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:16.403044 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c10ef4e-5871-43ee-8c2c-c3029bbeb5c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:16.403367 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c10ef4e-5871-43ee-8c2c-c3029bbeb5c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:16.414050 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c10ef4e-5871-43ee-8c2c-c3029bbeb5c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:28:16.416184 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c10ef4e-5871-43ee-8c2c-c3029bbeb5c2 service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:28:16.430432 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 882/1762] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:28:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:28:16.886986 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfae36d-c97d-44e8-a051-dabb54d647f1 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:28:16.888499 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-7bfae36d-c97d-44e8-a051-dabb54d647f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:28:16.903177 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfae36d-c97d-44e8-a051-dabb54d647f1 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:28:16.908949 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295a8fd1-01d3-481f-ad56-a76e720ed8c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:28:16.912164 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295a8fd1-01d3-481f-ad56-a76e720ed8c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:16.912164 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295a8fd1-01d3-481f-ad56-a76e720ed8c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:16.912164 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295a8fd1-01d3-481f-ad56-a76e720ed8c9 admin admin] PATH_INFO: `/v3/projects/60909f2642b94bdb8fae1b3621f1e2df` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:16.912164 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295a8fd1-01d3-481f-ad56-a76e720ed8c9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=60909f2642b94bdb8fae1b3621f1e2df)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:28:16.913242 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295a8fd1-01d3-481f-ad56-a76e720ed8c9 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:28:17.037708 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-295a8fd1-01d3-481f-ad56-a76e720ed8c9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '60909f2642b94bdb8fae1b3621f1e2df', 'request_id': 'req-295a8fd1-01d3-481f-ad56-a76e720ed8c9'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:28:17.044110 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-295a8fd1-01d3-481f-ad56-a76e720ed8c9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '60909f2642b94bdb8fae1b3621f1e2df', 'request_id': 'req-295a8fd1-01d3-481f-ad56-a76e720ed8c9'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:28:17.050897 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-295a8fd1-01d3-481f-ad56-a76e720ed8c9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '60909f2642b94bdb8fae1b3621f1e2df', 'request_id': 'req-295a8fd1-01d3-481f-ad56-a76e720ed8c9'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:28:17.159339 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-295a8fd1-01d3-481f-ad56-a76e720ed8c9 admin admin] The token cache is being invalidate because project 60909f2642b94bdb8fae1b3621f1e2df was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:28:17.159663 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-295a8fd1-01d3-481f-ad56-a76e720ed8c9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:28:17.160297 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-295a8fd1-01d3-481f-ad56-a76e720ed8c9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:28:17.161804 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 881/1763] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 24 09:28:16 2022] DELETE /identity/v3/projects/60909f2642b94bdb8fae1b3621f1e2df => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:28:17.166096 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c10ef4e-5871-43ee-8c2c-c3029bbeb5c2 service neutron] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:28:17.176842 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c10ef4e-5871-43ee-8c2c-c3029bbeb5c2 service neutron] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:28:17.236412 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09dcb66-9556-4856-8be9-58c3da5df2f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:28:17.237198 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09dcb66-9556-4856-8be9-58c3da5df2f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:17.237504 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09dcb66-9556-4856-8be9-58c3da5df2f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:17.237777 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09dcb66-9556-4856-8be9-58c3da5df2f3 admin admin] PATH_INFO: `/v3/users/caec843f49fa491db2b8c5565794940c` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:17.238903 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09dcb66-9556-4856-8be9-58c3da5df2f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=caec843f49fa491db2b8c5565794940c)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:28:17.240904 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09dcb66-9556-4856-8be9-58c3da5df2f3 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:28:17.485888 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-a09dcb66-9556-4856-8be9-58c3da5df2f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'caec843f49fa491db2b8c5565794940c'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:28:17.487764 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-a09dcb66-9556-4856-8be9-58c3da5df2f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'caec843f49fa491db2b8c5565794940c'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:28:17.621952 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-a09dcb66-9556-4856-8be9-58c3da5df2f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'caec843f49fa491db2b8c5565794940c'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:28:17.623310 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-a09dcb66-9556-4856-8be9-58c3da5df2f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'caec843f49fa491db2b8c5565794940c'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:28:17.631787 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-a09dcb66-9556-4856-8be9-58c3da5df2f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'caec843f49fa491db2b8c5565794940c'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:28:17.660621 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 883/1764] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 24 09:28:17 2022] DELETE /identity/v3/users/caec843f49fa491db2b8c5565794940c => generated 0 bytes in 495 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:28:18.230395 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295a8fd1-01d3-481f-ad56-a76e720ed8c9 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:28:18.232100 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-295a8fd1-01d3-481f-ad56-a76e720ed8c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:28:18.246652 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295a8fd1-01d3-481f-ad56-a76e720ed8c9 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:28:18.270398 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:28:18.332121 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8d646f-c72c-4d8f-a28a-2903387192fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:28:18.333103 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8d646f-c72c-4d8f-a28a-2903387192fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:18.334165 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8d646f-c72c-4d8f-a28a-2903387192fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:18.334622 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8d646f-c72c-4d8f-a28a-2903387192fb admin admin] PATH_INFO: `/v3/projects/b9c3da8e76534451952040e090603d0b` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:18.336555 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8d646f-c72c-4d8f-a28a-2903387192fb admin admin] RBAC: Authorizing `identity:delete_project(project_id=b9c3da8e76534451952040e090603d0b)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:28:18.339094 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8d646f-c72c-4d8f-a28a-2903387192fb admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:28:18.481348 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-6e8d646f-c72c-4d8f-a28a-2903387192fb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b9c3da8e76534451952040e090603d0b', 'request_id': 'req-6e8d646f-c72c-4d8f-a28a-2903387192fb'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:28:18.488480 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-6e8d646f-c72c-4d8f-a28a-2903387192fb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b9c3da8e76534451952040e090603d0b', 'request_id': 'req-6e8d646f-c72c-4d8f-a28a-2903387192fb'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:28:18.494181 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-6e8d646f-c72c-4d8f-a28a-2903387192fb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b9c3da8e76534451952040e090603d0b', 'request_id': 'req-6e8d646f-c72c-4d8f-a28a-2903387192fb'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:28:18.613143 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-6e8d646f-c72c-4d8f-a28a-2903387192fb admin admin] The token cache is being invalidate because project b9c3da8e76534451952040e090603d0b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:28:18.614232 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-6e8d646f-c72c-4d8f-a28a-2903387192fb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:28:18.614724 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-6e8d646f-c72c-4d8f-a28a-2903387192fb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:28:18.616492 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 882/1765] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 24 09:28:18 2022] DELETE /identity/v3/projects/b9c3da8e76534451952040e090603d0b => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:28:18.621348 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b9d318-a51c-4bf2-9675-f3f01db985d3 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:18.621861 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b9d318-a51c-4bf2-9675-f3f01db985d3 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:18.622103 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b9d318-a51c-4bf2-9675-f3f01db985d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:18.643211 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-27b9d318-a51c-4bf2-9675-f3f01db985d3 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:28:18.644595 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-27b9d318-a51c-4bf2-9675-f3f01db985d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:28:18.707252 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:28:18.726138 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 884/1766] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 09:28:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 09:28:18.734313 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01efc789-0e0a-4072-8ac8-11c248971165 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:18.734636 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01efc789-0e0a-4072-8ac8-11c248971165 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:18.734933 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01efc789-0e0a-4072-8ac8-11c248971165 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:18.756013 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-01efc789-0e0a-4072-8ac8-11c248971165 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:28:18.757702 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-01efc789-0e0a-4072-8ac8-11c248971165 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:28:18.765793 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 883/1767] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 09:28:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:28:18.770292 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b9d318-a51c-4bf2-9675-f3f01db985d3 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:28:18.781239 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b9d318-a51c-4bf2-9675-f3f01db985d3 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:28:18.787406 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade8747f-4780-4fb6-b73d-96cff1507a5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:28:18.788047 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade8747f-4780-4fb6-b73d-96cff1507a5f admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:18.788330 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade8747f-4780-4fb6-b73d-96cff1507a5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:18.788606 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade8747f-4780-4fb6-b73d-96cff1507a5f admin admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:18.789270 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade8747f-4780-4fb6-b73d-96cff1507a5f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:28:18.789578 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade8747f-4780-4fb6-b73d-96cff1507a5f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:28:18.791234 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade8747f-4780-4fb6-b73d-96cff1507a5f admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:28:18.803283 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 885/1768] 10.222.0.16 () {58 vars in 1153 bytes} [Wed Aug 24 09:28:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:28:18.807535 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01efc789-0e0a-4072-8ac8-11c248971165 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:28:18.816799 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01efc789-0e0a-4072-8ac8-11c248971165 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:28:18.821465 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c79d62-b3a5-4e25-a4aa-235f1e767e9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:28:18.822072 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c79d62-b3a5-4e25-a4aa-235f1e767e9d admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:18.822360 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c79d62-b3a5-4e25-a4aa-235f1e767e9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:18.822636 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c79d62-b3a5-4e25-a4aa-235f1e767e9d admin admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:18.823264 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c79d62-b3a5-4e25-a4aa-235f1e767e9d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:28:18.825255 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c79d62-b3a5-4e25-a4aa-235f1e767e9d admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:28:18.929295 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 884/1769] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 24 09:28:18 2022] POST /identity/v3/projects => generated 393 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:28:18.933600 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade8747f-4780-4fb6-b73d-96cff1507a5f admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:28:18.943684 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade8747f-4780-4fb6-b73d-96cff1507a5f admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:28:19.011462 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146150b2-e1df-4817-9515-1c124204199a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:28:19.012030 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146150b2-e1df-4817-9515-1c124204199a admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:19.012316 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146150b2-e1df-4817-9515-1c124204199a admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:19.012592 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146150b2-e1df-4817-9515-1c124204199a admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:19.013267 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146150b2-e1df-4817-9515-1c124204199a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:28:19.015292 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146150b2-e1df-4817-9515-1c124204199a admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:28:19.144075 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 886/1770] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 24 09:28:18 2022] POST /identity/v3/users => generated 297 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:28:19.148287 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c79d62-b3a5-4e25-a4aa-235f1e767e9d admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:28:19.160471 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c79d62-b3a5-4e25-a4aa-235f1e767e9d admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:28:19.178947 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac541d36-52eb-4be9-9dc1-3307b6f1ddda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:28:19.180024 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac541d36-52eb-4be9-9dc1-3307b6f1ddda admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:19.180024 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac541d36-52eb-4be9-9dc1-3307b6f1ddda admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:19.180255 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac541d36-52eb-4be9-9dc1-3307b6f1ddda admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:19.181061 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac541d36-52eb-4be9-9dc1-3307b6f1ddda admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:28:19.182728 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac541d36-52eb-4be9-9dc1-3307b6f1ddda admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:28:19.202055 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 885/1771] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 24 09:28:19 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:28:19.205191 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146150b2-e1df-4817-9515-1c124204199a admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:28:19.221356 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146150b2-e1df-4817-9515-1c124204199a admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:28:19.233471 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bad1653-dea3-4469-9010-c0afc690511a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:28:19.234477 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bad1653-dea3-4469-9010-c0afc690511a admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:19.234831 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bad1653-dea3-4469-9010-c0afc690511a admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:19.253588 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bad1653-dea3-4469-9010-c0afc690511a admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:19.253588 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bad1653-dea3-4469-9010-c0afc690511a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:28:19.268582 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bad1653-dea3-4469-9010-c0afc690511a admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:28:19.285591 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fb4b3a-8a95-4e92-af8b-31db5c00c317 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:19.289216 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fb4b3a-8a95-4e92-af8b-31db5c00c317 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:19.289216 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fb4b3a-8a95-4e92-af8b-31db5c00c317 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:19.313737 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 887/1772] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 24 09:28:19 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:28:19.315613 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bad1653-dea3-4469-9010-c0afc690511a admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:28:19.348146 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bad1653-dea3-4469-9010-c0afc690511a admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:28:19.353669 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81c462e-979f-45e9-9c3a-5398b9b83afe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:28:19.354895 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81c462e-979f-45e9-9c3a-5398b9b83afe admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:19.355529 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81c462e-979f-45e9-9c3a-5398b9b83afe admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:19.357352 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81c462e-979f-45e9-9c3a-5398b9b83afe admin admin] PATH_INFO: `/v3/projects/bf062c0dadc843e3957206ad5f75cdff/users/55f123efb16a44f0977287ba952093f3/roles/136cdf78e00a426997dd4e37faff489b` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:19.426590 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81c462e-979f-45e9-9c3a-5398b9b83afe admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf062c0dadc843e3957206ad5f75cdff, user_id=55f123efb16a44f0977287ba952093f3, role_id=136cdf78e00a426997dd4e37faff489b)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:28:19.439660 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81c462e-979f-45e9-9c3a-5398b9b83afe admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:28:19.480257 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-93fb4b3a-8a95-4e92-af8b-31db5c00c317 None None] MFA Rules not processed for user `0489462287fc4aea8e8e720af24e1ef8`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:28:19.488990 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-93fb4b3a-8a95-4e92-af8b-31db5c00c317 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:28:19.546570 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 888/1773] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 24 09:28:19 2022] PUT /identity/v3/projects/bf062c0dadc843e3957206ad5f75cdff/users/55f123efb16a44f0977287ba952093f3/roles/136cdf78e00a426997dd4e37faff489b => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:28:19.547174 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcdfc28-1dab-4176-8041-c650c89904f2 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:19.547456 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcdfc28-1dab-4176-8041-c650c89904f2 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:19.547782 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcdfc28-1dab-4176-8041-c650c89904f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:19.638273 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-afcdfc28-1dab-4176-8041-c650c89904f2 None None] MFA Rules not processed for user `8e5260e798404e8bb12b25ca8ea97052`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:28:19.639938 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-afcdfc28-1dab-4176-8041-c650c89904f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:28:19.682674 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:28:19.745273 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 886/1774] 10.222.0.16 () {58 vars in 1009 bytes} [Wed Aug 24 09:28:19 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 461 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:28:19.761142 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2b8e8c-6311-4533-bbd1-35af653665e0 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:19.761705 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2b8e8c-6311-4533-bbd1-35af653665e0 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:19.761990 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2b8e8c-6311-4533-bbd1-35af653665e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:19.867762 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 889/1775] 10.222.0.17 () {58 vars in 1015 bytes} [Wed Aug 24 09:28:19 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 321 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:28:19.884092 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-8b2b8e8c-6311-4533-bbd1-35af653665e0 None None] MFA Rules not processed for user `55f123efb16a44f0977287ba952093f3`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:28:19.884092 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-8b2b8e8c-6311-4533-bbd1-35af653665e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:28:19.916753 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcdfc28-1dab-4176-8041-c650c89904f2 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:28:19.918950 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-afcdfc28-1dab-4176-8041-c650c89904f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:28:19.938904 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcdfc28-1dab-4176-8041-c650c89904f2 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:28:19.955146 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d71c51b-a7c0-4b28-96a3-0098754e9dac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e5260e798404e8bb12b25ca8ea97052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:28:19.959016 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d71c51b-a7c0-4b28-96a3-0098754e9dac service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:19.959016 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d71c51b-a7c0-4b28-96a3-0098754e9dac service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:19.959016 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d71c51b-a7c0-4b28-96a3-0098754e9dac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:19.988503 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d71c51b-a7c0-4b28-96a3-0098754e9dac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:28:19.992638 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d71c51b-a7c0-4b28-96a3-0098754e9dac service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:28:20.013675 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 890/1776] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:28:19 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 09:28:20.116095 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 887/1777] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 09:28:19 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 369 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:28:20.120817 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d71c51b-a7c0-4b28-96a3-0098754e9dac service neutron] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:28:20.186779 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:28:20.232330 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d71c51b-a7c0-4b28-96a3-0098754e9dac service neutron] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:28:20.238347 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083c376a-a893-49aa-9cb8-125e99c6ec77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:28:20.239088 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083c376a-a893-49aa-9cb8-125e99c6ec77 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:20.239476 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083c376a-a893-49aa-9cb8-125e99c6ec77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:20.239882 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083c376a-a893-49aa-9cb8-125e99c6ec77 admin admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:20.240714 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083c376a-a893-49aa-9cb8-125e99c6ec77 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:28:20.243906 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083c376a-a893-49aa-9cb8-125e99c6ec77 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:28:20.312264 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2b8e8c-6311-4533-bbd1-35af653665e0 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:28:20.314132 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-8b2b8e8c-6311-4533-bbd1-35af653665e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:28:20.335776 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 891/1778] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 24 09:28:20 2022] POST /identity/v3/projects => generated 393 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:28:20.342186 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083c376a-a893-49aa-9cb8-125e99c6ec77 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:28:20.478341 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2b8e8c-6311-4533-bbd1-35af653665e0 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:28:20.509170 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083c376a-a893-49aa-9cb8-125e99c6ec77 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:28:20.521957 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe791d6-0b50-44c8-868a-87232bbbd377 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:28:20.524959 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe791d6-0b50-44c8-868a-87232bbbd377 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:20.525192 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe791d6-0b50-44c8-868a-87232bbbd377 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:20.527200 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe791d6-0b50-44c8-868a-87232bbbd377 admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:20.528305 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe791d6-0b50-44c8-868a-87232bbbd377 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:28:20.537934 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe791d6-0b50-44c8-868a-87232bbbd377 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:28:20.634024 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:28:20.655552 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35f03b5-2a3c-4813-b9d5-6f78852da5ed service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdd5f0d67fe54e9fae6f4af644988d93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:28:20.655889 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35f03b5-2a3c-4813-b9d5-6f78852da5ed service swift] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:20.656184 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35f03b5-2a3c-4813-b9d5-6f78852da5ed service swift] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:20.656470 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35f03b5-2a3c-4813-b9d5-6f78852da5ed service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:20.681817 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 892/1779] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 24 09:28:20 2022] POST /identity/v3/users => generated 303 bytes in 341 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:28:20.686573 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe791d6-0b50-44c8-868a-87232bbbd377 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:28:20.711520 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe791d6-0b50-44c8-868a-87232bbbd377 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:28:20.719099 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c7484f-a8ee-4b3d-99b5-95787076c9ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:28:20.720220 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c7484f-a8ee-4b3d-99b5-95787076c9ba admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:20.720756 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c7484f-a8ee-4b3d-99b5-95787076c9ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:20.721308 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c7484f-a8ee-4b3d-99b5-95787076c9ba admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:20.722288 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c7484f-a8ee-4b3d-99b5-95787076c9ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:28:20.724078 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c7484f-a8ee-4b3d-99b5-95787076c9ba admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:28:20.742623 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 893/1780] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 24 09:28:20 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 09:28:20.746327 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c7484f-a8ee-4b3d-99b5-95787076c9ba admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:28:20.770610 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c7484f-a8ee-4b3d-99b5-95787076c9ba admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:28:20.776087 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35f03b5-2a3c-4813-b9d5-6f78852da5ed service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:28:20.777708 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35f03b5-2a3c-4813-b9d5-6f78852da5ed service swift] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:28:20.787100 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb9995f-6f9b-470c-ac8d-e5845efa9c93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:28:20.787876 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb9995f-6f9b-470c-ac8d-e5845efa9c93 admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:20.788100 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb9995f-6f9b-470c-ac8d-e5845efa9c93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:20.788389 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb9995f-6f9b-470c-ac8d-e5845efa9c93 admin admin] PATH_INFO: `/v3/projects/8b6a9981f5624bd684b05d4f2c54af85/users/4c992e1f31034e0a9422050aba995b96/roles/15f634d4a2d247ddab90c458e5e1c141` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:20.803132 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 888/1781] 10.222.0.16 () {60 vars in 1384 bytes} [Wed Aug 24 09:28:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 491 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 24 09:28:20.824596 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb9995f-6f9b-470c-ac8d-e5845efa9c93 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b6a9981f5624bd684b05d4f2c54af85, user_id=4c992e1f31034e0a9422050aba995b96, role_id=15f634d4a2d247ddab90c458e5e1c141)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:28:20.828809 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb9995f-6f9b-470c-ac8d-e5845efa9c93 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:28:20.944647 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 894/1782] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 24 09:28:20 2022] PUT /identity/v3/projects/8b6a9981f5624bd684b05d4f2c54af85/users/4c992e1f31034e0a9422050aba995b96/roles/15f634d4a2d247ddab90c458e5e1c141 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:28:20.948675 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35f03b5-2a3c-4813-b9d5-6f78852da5ed service swift] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:28:20.972964 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35f03b5-2a3c-4813-b9d5-6f78852da5ed service swift] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:28:21.181461 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe5352b-30b2-4d81-8330-d9b0fec314ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:28:21.182505 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe5352b-30b2-4d81-8330-d9b0fec314ad admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:21.182939 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe5352b-30b2-4d81-8330-d9b0fec314ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:21.183362 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe5352b-30b2-4d81-8330-d9b0fec314ad admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:21.184401 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe5352b-30b2-4d81-8330-d9b0fec314ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:28:21.189038 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe5352b-30b2-4d81-8330-d9b0fec314ad admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:28:21.204108 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 889/1783] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 24 09:28:20 2022] GET /identity/v3/roles => generated 1420 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 09:28:21.208375 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb9995f-6f9b-470c-ac8d-e5845efa9c93 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:28:21.240178 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb9995f-6f9b-470c-ac8d-e5845efa9c93 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:28:21.247608 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcec6ad-9179-44a3-b645-7eb90d6ccd15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:28:21.248361 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcec6ad-9179-44a3-b645-7eb90d6ccd15 admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:21.248895 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcec6ad-9179-44a3-b645-7eb90d6ccd15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:21.249441 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcec6ad-9179-44a3-b645-7eb90d6ccd15 admin admin] PATH_INFO: `/v3/projects/8b6a9981f5624bd684b05d4f2c54af85/users/4c992e1f31034e0a9422050aba995b96/roles/15f634d4a2d247ddab90c458e5e1c141` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:21.251826 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcec6ad-9179-44a3-b645-7eb90d6ccd15 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b6a9981f5624bd684b05d4f2c54af85, user_id=4c992e1f31034e0a9422050aba995b96, role_id=15f634d4a2d247ddab90c458e5e1c141)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:28:21.256047 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcec6ad-9179-44a3-b645-7eb90d6ccd15 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:28:21.272325 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 895/1784] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 24 09:28:21 2022] PUT /identity/v3/projects/8b6a9981f5624bd684b05d4f2c54af85/users/4c992e1f31034e0a9422050aba995b96/roles/15f634d4a2d247ddab90c458e5e1c141 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:28:21.283123 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343748f9-29f5-49a1-8f48-196fde7b5325 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:21.284485 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343748f9-29f5-49a1-8f48-196fde7b5325 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:21.284910 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343748f9-29f5-49a1-8f48-196fde7b5325 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:21.399926 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-343748f9-29f5-49a1-8f48-196fde7b5325 None None] MFA Rules not processed for user `4c992e1f31034e0a9422050aba995b96`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:28:21.402328 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-343748f9-29f5-49a1-8f48-196fde7b5325 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:28:21.605786 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 890/1785] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 09:28:21 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 324 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:28:21.623222 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcec6ad-9179-44a3-b645-7eb90d6ccd15 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:28:21.623222 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-9dcec6ad-9179-44a3-b645-7eb90d6ccd15 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:28:21.786918 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcec6ad-9179-44a3-b645-7eb90d6ccd15 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:28:21.794545 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6aa06e-0a0a-4737-a29b-adb1a416e0b6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c32b83577c46488906fe409a790b24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:28:21.795820 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6aa06e-0a0a-4737-a29b-adb1a416e0b6 service cinder] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:21.796799 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6aa06e-0a0a-4737-a29b-adb1a416e0b6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:21.797280 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6aa06e-0a0a-4737-a29b-adb1a416e0b6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:21.808709 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:28:21.812318 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6aa06e-0a0a-4737-a29b-adb1a416e0b6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:28:21.814725 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6aa06e-0a0a-4737-a29b-adb1a416e0b6 service cinder] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:28:21.830835 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 896/1786] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:28:21 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 09:28:21.924110 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343748f9-29f5-49a1-8f48-196fde7b5325 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:28:21.925926 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-343748f9-29f5-49a1-8f48-196fde7b5325 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:28:21.959109 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343748f9-29f5-49a1-8f48-196fde7b5325 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:28:21.973696 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6141ad92-96bb-4c06-bcc0-ab6daf4c818b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c32b83577c46488906fe409a790b24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:28:21.976523 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6141ad92-96bb-4c06-bcc0-ab6daf4c818b service cinder] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:21.977523 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6141ad92-96bb-4c06-bcc0-ab6daf4c818b service cinder] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:21.977826 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6141ad92-96bb-4c06-bcc0-ab6daf4c818b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:22.079737 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:28:22.215043 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6141ad92-96bb-4c06-bcc0-ab6daf4c818b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:28:22.225933 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6141ad92-96bb-4c06-bcc0-ab6daf4c818b service cinder] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:28:22.264419 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 891/1787] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:28:21 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:28:22.365904 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a21514b-45a4-4f64-98e8-ab760e55f038 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:22.366553 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a21514b-45a4-4f64-98e8-ab760e55f038 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:22.367034 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a21514b-45a4-4f64-98e8-ab760e55f038 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:22.369045 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 897/1788] 10.222.0.16 () {54 vars in 921 bytes} [Wed Aug 24 09:28:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:28:22.450020 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f443813a-b0d7-465e-96b0-8f48630e399c None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:22.470685 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f443813a-b0d7-465e-96b0-8f48630e399c None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:22.470685 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f443813a-b0d7-465e-96b0-8f48630e399c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:22.493953 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-f443813a-b0d7-465e-96b0-8f48630e399c None None] MFA Rules not processed for user `1f055f7dc51d43bda9b315ca67f11d58`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:28:22.497170 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-f443813a-b0d7-465e-96b0-8f48630e399c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:28:22.702401 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 892/1789] 10.222.0.16 () {58 vars in 1018 bytes} [Wed Aug 24 09:28:22 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:28:22.804905 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a21514b-45a4-4f64-98e8-ab760e55f038 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:28:22.807690 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-3a21514b-45a4-4f64-98e8-ab760e55f038 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:28:22.867149 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a21514b-45a4-4f64-98e8-ab760e55f038 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:28:22.880740 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d8ed79-2407-4f56-bc6b-936aac1beb85 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f055f7dc51d43bda9b315ca67f11d58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:28:22.881428 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d8ed79-2407-4f56-bc6b-936aac1beb85 service nova] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:22.881919 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d8ed79-2407-4f56-bc6b-936aac1beb85 service nova] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:22.882348 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d8ed79-2407-4f56-bc6b-936aac1beb85 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:22.900826 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d8ed79-2407-4f56-bc6b-936aac1beb85 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:28:22.900826 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d8ed79-2407-4f56-bc6b-936aac1beb85 service nova] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:28:22.907528 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 898/1790] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:28:22 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:28:32.658009 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=50 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:28:32.900929 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=8 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:28:40.628364 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c166a2b5-5ff7-4205-9ab2-cfb2d93a4b18 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:40.629274 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c166a2b5-5ff7-4205-9ab2-cfb2d93a4b18 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:40.629696 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c166a2b5-5ff7-4205-9ab2-cfb2d93a4b18 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:40.634262 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 893/1791] 10.222.0.16 () {54 vars in 921 bytes} [Wed Aug 24 09:28:40 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:28:40.659689 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42120814-8381-4057-a11b-85369b05d656 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:40.661146 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42120814-8381-4057-a11b-85369b05d656 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:40.661588 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42120814-8381-4057-a11b-85369b05d656 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:40.687514 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-42120814-8381-4057-a11b-85369b05d656 None None] MFA Rules not processed for user `1f055f7dc51d43bda9b315ca67f11d58`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:28:40.689108 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-42120814-8381-4057-a11b-85369b05d656 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:28:40.701414 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 899/1792] 10.222.0.16 () {58 vars in 1018 bytes} [Wed Aug 24 09:28:40 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:28:40.725401 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c166a2b5-5ff7-4205-9ab2-cfb2d93a4b18 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:28:40.754106 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c166a2b5-5ff7-4205-9ab2-cfb2d93a4b18 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:28:40.770396 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0f6ae3-6ed3-4d1a-a9b2-6bde61706495 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f055f7dc51d43bda9b315ca67f11d58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:28:40.779882 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0f6ae3-6ed3-4d1a-a9b2-6bde61706495 service nova] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:40.780386 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0f6ae3-6ed3-4d1a-a9b2-6bde61706495 service nova] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:40.780982 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0f6ae3-6ed3-4d1a-a9b2-6bde61706495 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:40.803434 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42120814-8381-4057-a11b-85369b05d656 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:28:40.817192 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0f6ae3-6ed3-4d1a-a9b2-6bde61706495 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:28:40.821710 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0f6ae3-6ed3-4d1a-a9b2-6bde61706495 service nova] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:28:40.864553 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 894/1793] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:28:40 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:28:40.994079 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42120814-8381-4057-a11b-85369b05d656 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:28:41.002337 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869c3592-5411-478d-857a-d8ba39f6262c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:28:41.003089 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869c3592-5411-478d-857a-d8ba39f6262c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:41.003479 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869c3592-5411-478d-857a-d8ba39f6262c admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:41.003853 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869c3592-5411-478d-857a-d8ba39f6262c admin admin] PATH_INFO: `/v3/users/55f123efb16a44f0977287ba952093f3` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:41.005232 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869c3592-5411-478d-857a-d8ba39f6262c admin admin] RBAC: Authorizing `identity:delete_user(user_id=55f123efb16a44f0977287ba952093f3)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:28:41.007394 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869c3592-5411-478d-857a-d8ba39f6262c admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:28:41.296904 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-869c3592-5411-478d-857a-d8ba39f6262c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '55f123efb16a44f0977287ba952093f3'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:28:41.297846 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-869c3592-5411-478d-857a-d8ba39f6262c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '55f123efb16a44f0977287ba952093f3'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:28:41.381725 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0f6ae3-6ed3-4d1a-a9b2-6bde61706495 service nova] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:28:41.383155 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-bd0f6ae3-6ed3-4d1a-a9b2-6bde61706495 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:28:41.434333 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-869c3592-5411-478d-857a-d8ba39f6262c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '55f123efb16a44f0977287ba952093f3'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:28:41.435000 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-869c3592-5411-478d-857a-d8ba39f6262c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '55f123efb16a44f0977287ba952093f3'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:28:41.444501 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0f6ae3-6ed3-4d1a-a9b2-6bde61706495 service nova] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:28:41.450398 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-869c3592-5411-478d-857a-d8ba39f6262c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '55f123efb16a44f0977287ba952093f3'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:28:41.455898 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368959ad-1257-4cce-b1c5-040c0dc579ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:28:41.457231 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368959ad-1257-4cce-b1c5-040c0dc579ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:41.457581 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368959ad-1257-4cce-b1c5-040c0dc579ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:41.457923 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368959ad-1257-4cce-b1c5-040c0dc579ff admin admin] PATH_INFO: `/v3/users/30d814217f074fd7a57cef02ceddc31f` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:41.459200 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368959ad-1257-4cce-b1c5-040c0dc579ff admin admin] RBAC: Authorizing `identity:delete_user(user_id=30d814217f074fd7a57cef02ceddc31f)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:28:41.461206 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368959ad-1257-4cce-b1c5-040c0dc579ff admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:28:41.474489 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 900/1794] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 24 09:28:40 2022] DELETE /identity/v3/users/55f123efb16a44f0977287ba952093f3 => generated 0 bytes in 672 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 09:28:41.478925 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb339f3-e905-4569-a9ff-b0e8e0fd9770 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:41.479218 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb339f3-e905-4569-a9ff-b0e8e0fd9770 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:41.479501 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb339f3-e905-4569-a9ff-b0e8e0fd9770 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:41.500988 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-6fb339f3-e905-4569-a9ff-b0e8e0fd9770 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:28:41.502237 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-6fb339f3-e905-4569-a9ff-b0e8e0fd9770 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:28:41.565046 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 901/1795] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 09:28:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 09:28:41.581716 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb339f3-e905-4569-a9ff-b0e8e0fd9770 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:28:41.583252 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-6fb339f3-e905-4569-a9ff-b0e8e0fd9770 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:28:41.603098 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:28:41.650222 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb339f3-e905-4569-a9ff-b0e8e0fd9770 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:28:41.653877 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664cd153-fbff-4712-8974-fdde2cad6f78 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e5260e798404e8bb12b25ca8ea97052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:28:41.654629 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664cd153-fbff-4712-8974-fdde2cad6f78 service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:41.654910 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664cd153-fbff-4712-8974-fdde2cad6f78 service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:41.655455 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664cd153-fbff-4712-8974-fdde2cad6f78 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:41.665415 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664cd153-fbff-4712-8974-fdde2cad6f78 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:28:41.669891 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664cd153-fbff-4712-8974-fdde2cad6f78 service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:28:41.679992 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 902/1796] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:28:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:28:41.689367 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-368959ad-1257-4cce-b1c5-040c0dc579ff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '30d814217f074fd7a57cef02ceddc31f'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:28:41.690381 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-368959ad-1257-4cce-b1c5-040c0dc579ff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '30d814217f074fd7a57cef02ceddc31f'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:28:41.706049 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-368959ad-1257-4cce-b1c5-040c0dc579ff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '30d814217f074fd7a57cef02ceddc31f'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:28:41.715432 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-368959ad-1257-4cce-b1c5-040c0dc579ff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '30d814217f074fd7a57cef02ceddc31f'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:28:41.764940 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-368959ad-1257-4cce-b1c5-040c0dc579ff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '30d814217f074fd7a57cef02ceddc31f'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:28:41.771640 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 895/1797] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 24 09:28:41 2022] DELETE /identity/v3/users/30d814217f074fd7a57cef02ceddc31f => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 09:28:41.776306 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a70ca27-0339-443c-bb6f-ea08e4d12b01 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:41.776969 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a70ca27-0339-443c-bb6f-ea08e4d12b01 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:41.777784 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a70ca27-0339-443c-bb6f-ea08e4d12b01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:41.814331 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-9a70ca27-0339-443c-bb6f-ea08e4d12b01 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:28:41.815751 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-9a70ca27-0339-443c-bb6f-ea08e4d12b01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:28:41.880115 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 903/1798] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 09:28:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:28:41.890064 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368959ad-1257-4cce-b1c5-040c0dc579ff admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:28:41.891470 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-368959ad-1257-4cce-b1c5-040c0dc579ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:28:41.958078 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368959ad-1257-4cce-b1c5-040c0dc579ff admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:28:41.961993 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48824894-06ab-478c-aa90-f6ed1ff9071b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e5260e798404e8bb12b25ca8ea97052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:28:41.962606 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48824894-06ab-478c-aa90-f6ed1ff9071b service neutron] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:41.962907 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48824894-06ab-478c-aa90-f6ed1ff9071b service neutron] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:41.963722 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48824894-06ab-478c-aa90-f6ed1ff9071b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:41.975866 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48824894-06ab-478c-aa90-f6ed1ff9071b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:28:41.978369 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48824894-06ab-478c-aa90-f6ed1ff9071b service neutron] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:28:41.987274 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 896/1799] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:28:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:28:42.164813 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a70ca27-0339-443c-bb6f-ea08e4d12b01 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:28:42.164813 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-9a70ca27-0339-443c-bb6f-ea08e4d12b01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:28:42.185435 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a70ca27-0339-443c-bb6f-ea08e4d12b01 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:28:42.201933 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33769ae1-abd6-4173-a854-9f9884e17c74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:28:42.202696 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33769ae1-abd6-4173-a854-9f9884e17c74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:42.202991 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33769ae1-abd6-4173-a854-9f9884e17c74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:42.203269 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33769ae1-abd6-4173-a854-9f9884e17c74 admin admin] PATH_INFO: `/v3/projects/bf062c0dadc843e3957206ad5f75cdff` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:42.209147 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33769ae1-abd6-4173-a854-9f9884e17c74 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bf062c0dadc843e3957206ad5f75cdff)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:28:42.211104 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33769ae1-abd6-4173-a854-9f9884e17c74 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:28:42.338860 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-33769ae1-abd6-4173-a854-9f9884e17c74 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bf062c0dadc843e3957206ad5f75cdff', 'request_id': 'req-33769ae1-abd6-4173-a854-9f9884e17c74'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:28:42.346066 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-33769ae1-abd6-4173-a854-9f9884e17c74 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bf062c0dadc843e3957206ad5f75cdff', 'request_id': 'req-33769ae1-abd6-4173-a854-9f9884e17c74'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:28:42.353493 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-33769ae1-abd6-4173-a854-9f9884e17c74 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bf062c0dadc843e3957206ad5f75cdff', 'request_id': 'req-33769ae1-abd6-4173-a854-9f9884e17c74'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:28:42.416754 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-33769ae1-abd6-4173-a854-9f9884e17c74 admin admin] The token cache is being invalidate because project bf062c0dadc843e3957206ad5f75cdff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81602) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:28:42.416966 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-33769ae1-abd6-4173-a854-9f9884e17c74 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:28:42.418129 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-33769ae1-abd6-4173-a854-9f9884e17c74 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:28:42.422794 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 904/1800] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 24 09:28:42 2022] DELETE /identity/v3/projects/bf062c0dadc843e3957206ad5f75cdff => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:28:42.423817 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48824894-06ab-478c-aa90-f6ed1ff9071b service neutron] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:28:42.434581 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48824894-06ab-478c-aa90-f6ed1ff9071b service neutron] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:28:42.495865 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277ad476-42ee-474f-a487-363510d11ab9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:28:42.496512 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277ad476-42ee-474f-a487-363510d11ab9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:42.496796 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277ad476-42ee-474f-a487-363510d11ab9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:42.497106 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277ad476-42ee-474f-a487-363510d11ab9 admin admin] PATH_INFO: `/v3/users/4c992e1f31034e0a9422050aba995b96` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:42.499136 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277ad476-42ee-474f-a487-363510d11ab9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4c992e1f31034e0a9422050aba995b96)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:28:42.501198 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277ad476-42ee-474f-a487-363510d11ab9 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:28:42.503720 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:28:42.515778 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33769ae1-abd6-4173-a854-9f9884e17c74 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:28:42.517851 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-33769ae1-abd6-4173-a854-9f9884e17c74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:28:42.533366 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33769ae1-abd6-4173-a854-9f9884e17c74 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:28:42.538324 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cd127d-e83b-4cb5-b782-67941613b1f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:28:42.539145 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cd127d-e83b-4cb5-b782-67941613b1f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:42.539435 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cd127d-e83b-4cb5-b782-67941613b1f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:42.539846 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cd127d-e83b-4cb5-b782-67941613b1f7 admin admin] PATH_INFO: `/v3/projects/f613b5ff3b6649868a283bc063e9a978` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:42.541168 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cd127d-e83b-4cb5-b782-67941613b1f7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f613b5ff3b6649868a283bc063e9a978)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:28:42.543381 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cd127d-e83b-4cb5-b782-67941613b1f7 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:28:42.724653 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:28:42.735748 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-d5cd127d-e83b-4cb5-b782-67941613b1f7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f613b5ff3b6649868a283bc063e9a978', 'request_id': 'req-d5cd127d-e83b-4cb5-b782-67941613b1f7'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:28:42.741159 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-277ad476-42ee-474f-a487-363510d11ab9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4c992e1f31034e0a9422050aba995b96'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:28:42.742697 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-277ad476-42ee-474f-a487-363510d11ab9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4c992e1f31034e0a9422050aba995b96'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:28:42.743087 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-d5cd127d-e83b-4cb5-b782-67941613b1f7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f613b5ff3b6649868a283bc063e9a978', 'request_id': 'req-d5cd127d-e83b-4cb5-b782-67941613b1f7'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:28:42.750322 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-d5cd127d-e83b-4cb5-b782-67941613b1f7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f613b5ff3b6649868a283bc063e9a978', 'request_id': 'req-d5cd127d-e83b-4cb5-b782-67941613b1f7'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:28:42.760710 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-277ad476-42ee-474f-a487-363510d11ab9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4c992e1f31034e0a9422050aba995b96'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:28:42.767984 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-277ad476-42ee-474f-a487-363510d11ab9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4c992e1f31034e0a9422050aba995b96'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:28:42.823347 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-d5cd127d-e83b-4cb5-b782-67941613b1f7 admin admin] The token cache is being invalidate because project f613b5ff3b6649868a283bc063e9a978 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81602) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:28:42.824111 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-d5cd127d-e83b-4cb5-b782-67941613b1f7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:28:42.824989 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-d5cd127d-e83b-4cb5-b782-67941613b1f7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:28:42.827716 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 905/1801] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 24 09:28:42 2022] DELETE /identity/v3/projects/f613b5ff3b6649868a283bc063e9a978 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 09:28:42.831717 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cd127d-e83b-4cb5-b782-67941613b1f7 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:28:42.836915 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-d5cd127d-e83b-4cb5-b782-67941613b1f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:28:42.901480 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-277ad476-42ee-474f-a487-363510d11ab9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4c992e1f31034e0a9422050aba995b96'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:28:42.911287 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 897/1802] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 24 09:28:42 2022] DELETE /identity/v3/users/4c992e1f31034e0a9422050aba995b96 => generated 0 bytes in 488 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 09:28:42.941773 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cd127d-e83b-4cb5-b782-67941613b1f7 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:28:43.033419 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2085fe40-00ed-40aa-8665-628fffadce92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:28:43.034082 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2085fe40-00ed-40aa-8665-628fffadce92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:43.034364 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2085fe40-00ed-40aa-8665-628fffadce92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:43.034638 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2085fe40-00ed-40aa-8665-628fffadce92 admin admin] PATH_INFO: `/v3/users/a87941c110c049c4becad18bf3b09d94` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:43.036864 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2085fe40-00ed-40aa-8665-628fffadce92 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a87941c110c049c4becad18bf3b09d94)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:28:43.038723 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2085fe40-00ed-40aa-8665-628fffadce92 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:28:43.242918 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-2085fe40-00ed-40aa-8665-628fffadce92 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a87941c110c049c4becad18bf3b09d94'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:28:43.243626 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-2085fe40-00ed-40aa-8665-628fffadce92 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a87941c110c049c4becad18bf3b09d94'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:28:43.324068 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-2085fe40-00ed-40aa-8665-628fffadce92 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a87941c110c049c4becad18bf3b09d94'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:28:43.324693 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-2085fe40-00ed-40aa-8665-628fffadce92 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a87941c110c049c4becad18bf3b09d94'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:28:43.333326 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-2085fe40-00ed-40aa-8665-628fffadce92 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a87941c110c049c4becad18bf3b09d94'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:28:43.341249 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:28:43.351992 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 906/1803] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 24 09:28:42 2022] DELETE /identity/v3/users/a87941c110c049c4becad18bf3b09d94 => generated 0 bytes in 521 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 09:28:43.379680 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277ad476-42ee-474f-a487-363510d11ab9 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:28:43.381571 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-277ad476-42ee-474f-a487-363510d11ab9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:28:43.400096 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277ad476-42ee-474f-a487-363510d11ab9 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:28:43.482475 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5db092-a983-4bd9-b94c-de2743357ef4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:28:43.483034 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5db092-a983-4bd9-b94c-de2743357ef4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:43.483308 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5db092-a983-4bd9-b94c-de2743357ef4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:43.483593 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5db092-a983-4bd9-b94c-de2743357ef4 admin admin] PATH_INFO: `/v3/projects/8b6a9981f5624bd684b05d4f2c54af85` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:43.484905 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5db092-a983-4bd9-b94c-de2743357ef4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8b6a9981f5624bd684b05d4f2c54af85)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:28:43.486709 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5db092-a983-4bd9-b94c-de2743357ef4 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:28:43.618968 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-3d5db092-a983-4bd9-b94c-de2743357ef4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8b6a9981f5624bd684b05d4f2c54af85', 'request_id': 'req-3d5db092-a983-4bd9-b94c-de2743357ef4'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:28:43.626736 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-3d5db092-a983-4bd9-b94c-de2743357ef4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8b6a9981f5624bd684b05d4f2c54af85', 'request_id': 'req-3d5db092-a983-4bd9-b94c-de2743357ef4'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:28:43.637260 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-3d5db092-a983-4bd9-b94c-de2743357ef4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8b6a9981f5624bd684b05d4f2c54af85', 'request_id': 'req-3d5db092-a983-4bd9-b94c-de2743357ef4'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:28:43.701714 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-3d5db092-a983-4bd9-b94c-de2743357ef4 admin admin] The token cache is being invalidate because project 8b6a9981f5624bd684b05d4f2c54af85 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:28:43.702204 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-3d5db092-a983-4bd9-b94c-de2743357ef4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:28:43.702932 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-3d5db092-a983-4bd9-b94c-de2743357ef4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:28:43.707854 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 898/1804] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 24 09:28:43 2022] DELETE /identity/v3/projects/8b6a9981f5624bd684b05d4f2c54af85 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:28:43.710229 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a9dd62-90ae-4fe9-8597-0cafbf20e1cf None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:43.710524 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a9dd62-90ae-4fe9-8597-0cafbf20e1cf None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:43.710803 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a9dd62-90ae-4fe9-8597-0cafbf20e1cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:43.734378 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-28a9dd62-90ae-4fe9-8597-0cafbf20e1cf None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:28:43.735527 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-28a9dd62-90ae-4fe9-8597-0cafbf20e1cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:28:43.795329 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 907/1805] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 09:28:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 09:28:43.804632 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058f40d9-fa4f-4691-b71b-5fde980b808e None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:43.805061 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058f40d9-fa4f-4691-b71b-5fde980b808e None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:43.805190 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058f40d9-fa4f-4691-b71b-5fde980b808e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:43.807755 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a9dd62-90ae-4fe9-8597-0cafbf20e1cf None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:28:43.809328 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-28a9dd62-90ae-4fe9-8597-0cafbf20e1cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:28:43.822308 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a9dd62-90ae-4fe9-8597-0cafbf20e1cf None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:28:43.826759 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-058f40d9-fa4f-4691-b71b-5fde980b808e None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:28:43.827449 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cff6e04-52f9-4320-ac66-9ebd465ee436 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:28:43.828194 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cff6e04-52f9-4320-ac66-9ebd465ee436 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:43.828328 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-058f40d9-fa4f-4691-b71b-5fde980b808e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:28:43.828590 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cff6e04-52f9-4320-ac66-9ebd465ee436 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:43.828965 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cff6e04-52f9-4320-ac66-9ebd465ee436 admin admin] PATH_INFO: `/v3/projects/44a12461a70846a7aafc03f342c074a3` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:43.830164 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cff6e04-52f9-4320-ac66-9ebd465ee436 admin admin] RBAC: Authorizing `identity:delete_project(project_id=44a12461a70846a7aafc03f342c074a3)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:28:43.832205 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cff6e04-52f9-4320-ac66-9ebd465ee436 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:28:43.837156 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 899/1806] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 09:28:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:28:43.841986 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058f40d9-fa4f-4691-b71b-5fde980b808e None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:28:43.848146 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:28:43.852436 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058f40d9-fa4f-4691-b71b-5fde980b808e None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:28:43.858065 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2aec645-5ffe-4125-afce-2e3e73c70001 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:28:43.858763 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2aec645-5ffe-4125-afce-2e3e73c70001 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:43.859047 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2aec645-5ffe-4125-afce-2e3e73c70001 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:43.859323 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2aec645-5ffe-4125-afce-2e3e73c70001 admin admin] PATH_INFO: `/v3/domains` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:43.861100 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2aec645-5ffe-4125-afce-2e3e73c70001 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:28:43.861100 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2aec645-5ffe-4125-afce-2e3e73c70001 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:28:43.862080 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2aec645-5ffe-4125-afce-2e3e73c70001 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:28:43.874290 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 900/1807] 10.222.0.16 () {58 vars in 1153 bytes} [Wed Aug 24 09:28:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:28:43.878728 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2aec645-5ffe-4125-afce-2e3e73c70001 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:28:43.888734 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2aec645-5ffe-4125-afce-2e3e73c70001 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:28:43.893446 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2a326e-965b-4c90-a536-2838409e24e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:28:43.895626 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2a326e-965b-4c90-a536-2838409e24e8 admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:43.895626 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2a326e-965b-4c90-a536-2838409e24e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:43.895626 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2a326e-965b-4c90-a536-2838409e24e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:43.895626 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2a326e-965b-4c90-a536-2838409e24e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:28:43.897106 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2a326e-965b-4c90-a536-2838409e24e8 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:28:43.925181 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-1cff6e04-52f9-4320-ac66-9ebd465ee436 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '44a12461a70846a7aafc03f342c074a3', 'request_id': 'req-1cff6e04-52f9-4320-ac66-9ebd465ee436'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:28:43.931332 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-1cff6e04-52f9-4320-ac66-9ebd465ee436 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '44a12461a70846a7aafc03f342c074a3', 'request_id': 'req-1cff6e04-52f9-4320-ac66-9ebd465ee436'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:28:43.937172 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-1cff6e04-52f9-4320-ac66-9ebd465ee436 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '44a12461a70846a7aafc03f342c074a3', 'request_id': 'req-1cff6e04-52f9-4320-ac66-9ebd465ee436'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:28:43.991994 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 901/1808] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 24 09:28:43 2022] POST /identity/v3/projects => generated 373 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 09:28:43.993590 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2a326e-965b-4c90-a536-2838409e24e8 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:28:44.021696 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-1cff6e04-52f9-4320-ac66-9ebd465ee436 admin admin] The token cache is being invalidate because project 44a12461a70846a7aafc03f342c074a3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81602) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:28:44.021990 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-1cff6e04-52f9-4320-ac66-9ebd465ee436 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:28:44.023285 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-1cff6e04-52f9-4320-ac66-9ebd465ee436 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:28:44.024396 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 908/1809] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 24 09:28:43 2022] DELETE /identity/v3/projects/44a12461a70846a7aafc03f342c074a3 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 09:28:44.029335 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b95a0c-46af-4914-8f8a-d9d3d5083527 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:44.029652 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b95a0c-46af-4914-8f8a-d9d3d5083527 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:44.029938 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b95a0c-46af-4914-8f8a-d9d3d5083527 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:44.052371 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-40b95a0c-46af-4914-8f8a-d9d3d5083527 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:28:44.053575 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-40b95a0c-46af-4914-8f8a-d9d3d5083527 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:28:44.064893 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2a326e-965b-4c90-a536-2838409e24e8 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:28:44.069774 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff020d4-83cb-4b70-9288-37800c591566 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:28:44.070706 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff020d4-83cb-4b70-9288-37800c591566 admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:44.070706 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff020d4-83cb-4b70-9288-37800c591566 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:44.071126 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff020d4-83cb-4b70-9288-37800c591566 admin admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:44.071794 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff020d4-83cb-4b70-9288-37800c591566 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:28:44.075047 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff020d4-83cb-4b70-9288-37800c591566 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:28:44.139556 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 902/1810] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 24 09:28:43 2022] POST /identity/v3/users => generated 287 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 09:28:44.141008 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff020d4-83cb-4b70-9288-37800c591566 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:28:44.142751 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-aff020d4-83cb-4b70-9288-37800c591566 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:28:44.147647 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 909/1811] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 09:28:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 09:28:44.155547 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4784796b-0704-454e-8512-7804f481be2c None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:44.155864 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4784796b-0704-454e-8512-7804f481be2c None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:44.155984 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff020d4-83cb-4b70-9288-37800c591566 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:28:44.156139 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4784796b-0704-454e-8512-7804f481be2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:44.161015 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4483067a-2cb7-44dc-88a1-4ddb7babc519 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:28:44.161634 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4483067a-2cb7-44dc-88a1-4ddb7babc519 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:44.161918 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4483067a-2cb7-44dc-88a1-4ddb7babc519 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:44.162188 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4483067a-2cb7-44dc-88a1-4ddb7babc519 admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:44.162980 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4483067a-2cb7-44dc-88a1-4ddb7babc519 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:28:44.165779 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4483067a-2cb7-44dc-88a1-4ddb7babc519 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:28:44.177709 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-4784796b-0704-454e-8512-7804f481be2c None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:28:44.181672 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 903/1812] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 24 09:28:44 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 09:28:44.182389 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-4784796b-0704-454e-8512-7804f481be2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:28:44.186902 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4483067a-2cb7-44dc-88a1-4ddb7babc519 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:28:44.191887 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 910/1813] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 09:28:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 09:28:44.197754 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4483067a-2cb7-44dc-88a1-4ddb7babc519 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:28:44.202333 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4784796b-0704-454e-8512-7804f481be2c None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:28:44.204075 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08855c3e-502c-4c27-b065-2ed72031effa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:28:44.204707 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08855c3e-502c-4c27-b065-2ed72031effa admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:44.204992 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08855c3e-502c-4c27-b065-2ed72031effa admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:44.205271 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08855c3e-502c-4c27-b065-2ed72031effa admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:44.205959 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08855c3e-502c-4c27-b065-2ed72031effa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:28:44.207704 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08855c3e-502c-4c27-b065-2ed72031effa admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:28:44.214443 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4784796b-0704-454e-8512-7804f481be2c None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:28:44.221181 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c962f253-ecb7-47b9-a8c9-5dca10102c57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:28:44.221579 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c962f253-ecb7-47b9-a8c9-5dca10102c57 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:44.221934 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c962f253-ecb7-47b9-a8c9-5dca10102c57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:44.222213 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c962f253-ecb7-47b9-a8c9-5dca10102c57 admin admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:44.222878 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c962f253-ecb7-47b9-a8c9-5dca10102c57 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:28:44.223183 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c962f253-ecb7-47b9-a8c9-5dca10102c57 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:28:44.224823 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c962f253-ecb7-47b9-a8c9-5dca10102c57 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:28:44.225860 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 904/1814] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 24 09:28:44 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:28:44.229351 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:28:44.232289 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08855c3e-502c-4c27-b065-2ed72031effa admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:28:44.241590 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 911/1815] 10.222.0.16 () {58 vars in 1153 bytes} [Wed Aug 24 09:28:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:28:44.242745 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c962f253-ecb7-47b9-a8c9-5dca10102c57 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:28:44.247554 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08855c3e-502c-4c27-b065-2ed72031effa admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:28:44.252680 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c962f253-ecb7-47b9-a8c9-5dca10102c57 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:28:44.256075 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed501a11-0862-4a91-90a7-8a0a3e695d12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:28:44.256075 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed501a11-0862-4a91-90a7-8a0a3e695d12 admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:44.256075 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed501a11-0862-4a91-90a7-8a0a3e695d12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:44.256075 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed501a11-0862-4a91-90a7-8a0a3e695d12 admin admin] PATH_INFO: `/v3/projects/7656789d530947a9b05a97e7b0993ff4/users/a01fa696ca6a42a1b1432a4a6f162b1d/roles/136cdf78e00a426997dd4e37faff489b` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:44.257540 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300a45fa-04e8-4d2f-9a29-40572398aa67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:28:44.258154 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300a45fa-04e8-4d2f-9a29-40572398aa67 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:44.258433 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300a45fa-04e8-4d2f-9a29-40572398aa67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:44.258704 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300a45fa-04e8-4d2f-9a29-40572398aa67 admin admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:44.259354 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300a45fa-04e8-4d2f-9a29-40572398aa67 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:28:44.261311 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300a45fa-04e8-4d2f-9a29-40572398aa67 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:28:44.285270 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed501a11-0862-4a91-90a7-8a0a3e695d12 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7656789d530947a9b05a97e7b0993ff4, user_id=a01fa696ca6a42a1b1432a4a6f162b1d, role_id=136cdf78e00a426997dd4e37faff489b)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:28:44.288082 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed501a11-0862-4a91-90a7-8a0a3e695d12 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:28:44.360600 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 912/1816] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 24 09:28:44 2022] POST /identity/v3/projects => generated 381 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:28:44.364389 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300a45fa-04e8-4d2f-9a29-40572398aa67 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:28:44.377527 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 905/1817] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 24 09:28:44 2022] PUT /identity/v3/projects/7656789d530947a9b05a97e7b0993ff4/users/a01fa696ca6a42a1b1432a4a6f162b1d/roles/136cdf78e00a426997dd4e37faff489b => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:28:44.382824 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9614c79c-0a88-4b82-9246-aa0592729ebf None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:44.383415 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9614c79c-0a88-4b82-9246-aa0592729ebf None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:44.383415 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9614c79c-0a88-4b82-9246-aa0592729ebf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:44.401776 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-9614c79c-0a88-4b82-9246-aa0592729ebf None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:28:44.402973 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-9614c79c-0a88-4b82-9246-aa0592729ebf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:28:44.427797 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300a45fa-04e8-4d2f-9a29-40572398aa67 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:28:44.462012 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 906/1818] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 09:28:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:28:44.472103 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9614c79c-0a88-4b82-9246-aa0592729ebf None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:28:44.473478 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-9614c79c-0a88-4b82-9246-aa0592729ebf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:28:44.486048 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c901a8-63d4-4252-a487-4b81059a38b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:28:44.486659 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c901a8-63d4-4252-a487-4b81059a38b9 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:44.486939 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c901a8-63d4-4252-a487-4b81059a38b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:44.487207 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c901a8-63d4-4252-a487-4b81059a38b9 admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:44.487863 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c901a8-63d4-4252-a487-4b81059a38b9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:28:44.488075 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:28:44.489761 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c901a8-63d4-4252-a487-4b81059a38b9 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:28:44.532122 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9614c79c-0a88-4b82-9246-aa0592729ebf None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:28:44.535786 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574e53da-b686-446f-87dd-ababe4369775 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e5260e798404e8bb12b25ca8ea97052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:28:44.536462 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574e53da-b686-446f-87dd-ababe4369775 service neutron] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:44.536722 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574e53da-b686-446f-87dd-ababe4369775 service neutron] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:44.536993 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574e53da-b686-446f-87dd-ababe4369775 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:44.545828 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574e53da-b686-446f-87dd-ababe4369775 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:28:44.547247 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574e53da-b686-446f-87dd-ababe4369775 service neutron] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:28:44.553334 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 913/1819] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 24 09:28:44 2022] POST /identity/v3/users => generated 291 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 09:28:44.557037 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 907/1820] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:28:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 09:28:44.557464 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c901a8-63d4-4252-a487-4b81059a38b9 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:28:44.567898 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c901a8-63d4-4252-a487-4b81059a38b9 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:28:44.572702 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d607f9e8-50bc-4118-ad2a-a7c30c79c5f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:28:44.573457 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d607f9e8-50bc-4118-ad2a-a7c30c79c5f5 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:44.573739 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d607f9e8-50bc-4118-ad2a-a7c30c79c5f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:44.574014 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d607f9e8-50bc-4118-ad2a-a7c30c79c5f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:44.576024 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d607f9e8-50bc-4118-ad2a-a7c30c79c5f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:28:44.576338 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d607f9e8-50bc-4118-ad2a-a7c30c79c5f5 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:28:44.586474 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 914/1821] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 24 09:28:44 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 09:28:44.591606 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574e53da-b686-446f-87dd-ababe4369775 service neutron] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:28:44.601336 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574e53da-b686-446f-87dd-ababe4369775 service neutron] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:28:44.606217 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead35b87-f14f-45f8-a085-ea64f9796ae0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:28:44.607508 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead35b87-f14f-45f8-a085-ea64f9796ae0 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:44.607834 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead35b87-f14f-45f8-a085-ea64f9796ae0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:44.608113 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead35b87-f14f-45f8-a085-ea64f9796ae0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:44.608758 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead35b87-f14f-45f8-a085-ea64f9796ae0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:28:44.610393 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead35b87-f14f-45f8-a085-ea64f9796ae0 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:28:44.622692 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 908/1822] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 24 09:28:44 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:28:44.625115 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d607f9e8-50bc-4118-ad2a-a7c30c79c5f5 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:28:44.635374 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d607f9e8-50bc-4118-ad2a-a7c30c79c5f5 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:28:44.640086 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f582b1ac-3d66-493b-b590-c20e2374df41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:28:44.640740 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f582b1ac-3d66-493b-b590-c20e2374df41 admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:44.641072 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f582b1ac-3d66-493b-b590-c20e2374df41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:44.641280 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f582b1ac-3d66-493b-b590-c20e2374df41 admin admin] PATH_INFO: `/v3/projects/646e6f6795d5490489a8c3b6c1e9d908/users/e2e5d81ba0694d6b8ea7edbd335ea4ad/roles/136cdf78e00a426997dd4e37faff489b` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:44.659214 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f582b1ac-3d66-493b-b590-c20e2374df41 admin admin] RBAC: Authorizing `identity:create_grant(project_id=646e6f6795d5490489a8c3b6c1e9d908, user_id=e2e5d81ba0694d6b8ea7edbd335ea4ad, role_id=136cdf78e00a426997dd4e37faff489b)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:28:44.661968 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f582b1ac-3d66-493b-b590-c20e2374df41 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:28:44.727754 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 915/1823] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 24 09:28:44 2022] PUT /identity/v3/projects/646e6f6795d5490489a8c3b6c1e9d908/users/e2e5d81ba0694d6b8ea7edbd335ea4ad/roles/136cdf78e00a426997dd4e37faff489b => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:28:44.740605 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4eafc53-6f57-46c7-bdcc-d164f111e823 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:44.740895 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4eafc53-6f57-46c7-bdcc-d164f111e823 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:44.741176 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4eafc53-6f57-46c7-bdcc-d164f111e823 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:44.790405 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-e4eafc53-6f57-46c7-bdcc-d164f111e823 None None] MFA Rules not processed for user `e2e5d81ba0694d6b8ea7edbd335ea4ad`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:28:44.791722 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-e4eafc53-6f57-46c7-bdcc-d164f111e823 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:28:44.874848 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 909/1824] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 09:28:44 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:28:44.879480 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f582b1ac-3d66-493b-b590-c20e2374df41 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:28:44.922821 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:28:44.952508 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f582b1ac-3d66-493b-b590-c20e2374df41 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:28:44.957143 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0f59af-2223-40f5-a8ef-c83611e734a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:28:44.957754 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0f59af-2223-40f5-a8ef-c83611e734a8 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:44.958038 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0f59af-2223-40f5-a8ef-c83611e734a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:44.958317 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0f59af-2223-40f5-a8ef-c83611e734a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:44.958942 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0f59af-2223-40f5-a8ef-c83611e734a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:28:44.960988 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0f59af-2223-40f5-a8ef-c83611e734a8 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:28:45.056982 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 916/1825] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 24 09:28:44 2022] POST /identity/v3/projects => generated 381 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:28:45.061545 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4eafc53-6f57-46c7-bdcc-d164f111e823 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:28:45.073032 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4eafc53-6f57-46c7-bdcc-d164f111e823 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:28:45.132483 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580c08d7-70d1-442f-803d-9aeba008c2ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:28:45.133111 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580c08d7-70d1-442f-803d-9aeba008c2ff admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:45.133391 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580c08d7-70d1-442f-803d-9aeba008c2ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:45.133660 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580c08d7-70d1-442f-803d-9aeba008c2ff admin admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:45.134417 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580c08d7-70d1-442f-803d-9aeba008c2ff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:28:45.136384 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580c08d7-70d1-442f-803d-9aeba008c2ff admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:28:45.198242 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 910/1826] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 24 09:28:45 2022] POST /identity/v3/users => generated 297 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:28:45.202826 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0f59af-2223-40f5-a8ef-c83611e734a8 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:28:45.213992 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0f59af-2223-40f5-a8ef-c83611e734a8 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:28:45.219361 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7e8373-5ceb-4c93-a294-666ed9afca65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:28:45.220023 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7e8373-5ceb-4c93-a294-666ed9afca65 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:45.224273 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7e8373-5ceb-4c93-a294-666ed9afca65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:45.224273 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7e8373-5ceb-4c93-a294-666ed9afca65 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:45.224273 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7e8373-5ceb-4c93-a294-666ed9afca65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:28:45.224273 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7e8373-5ceb-4c93-a294-666ed9afca65 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:28:45.234503 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 917/1827] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 24 09:28:45 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:28:45.238707 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580c08d7-70d1-442f-803d-9aeba008c2ff admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:28:45.249150 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580c08d7-70d1-442f-803d-9aeba008c2ff admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:28:45.254342 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293eff64-4566-42b0-9380-16762c5e96d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:28:45.255007 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293eff64-4566-42b0-9380-16762c5e96d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:45.255288 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293eff64-4566-42b0-9380-16762c5e96d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:45.255552 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293eff64-4566-42b0-9380-16762c5e96d1 admin admin] PATH_INFO: `/v3/projects/ef95389480e6457987790aad48952c0b/users/b99715cd2bf8446b841725bbd3da2abf/roles/15f634d4a2d247ddab90c458e5e1c141` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:45.271165 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293eff64-4566-42b0-9380-16762c5e96d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef95389480e6457987790aad48952c0b, user_id=b99715cd2bf8446b841725bbd3da2abf, role_id=15f634d4a2d247ddab90c458e5e1c141)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:28:45.273935 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293eff64-4566-42b0-9380-16762c5e96d1 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:28:45.372485 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:28:45.380429 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 911/1828] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 24 09:28:45 2022] PUT /identity/v3/projects/ef95389480e6457987790aad48952c0b/users/b99715cd2bf8446b841725bbd3da2abf/roles/15f634d4a2d247ddab90c458e5e1c141 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:28:45.384353 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7e8373-5ceb-4c93-a294-666ed9afca65 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:28:45.394860 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7e8373-5ceb-4c93-a294-666ed9afca65 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:28:45.466851 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64fca4d-6f02-4281-9d5c-431bb54e9f11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:28:45.467603 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64fca4d-6f02-4281-9d5c-431bb54e9f11 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:45.467978 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64fca4d-6f02-4281-9d5c-431bb54e9f11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:45.468313 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64fca4d-6f02-4281-9d5c-431bb54e9f11 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:45.468963 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64fca4d-6f02-4281-9d5c-431bb54e9f11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:28:45.470787 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64fca4d-6f02-4281-9d5c-431bb54e9f11 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:28:45.481170 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 918/1829] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 24 09:28:45 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 09:28:45.486331 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293eff64-4566-42b0-9380-16762c5e96d1 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:28:45.496680 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293eff64-4566-42b0-9380-16762c5e96d1 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:28:45.501936 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b62cb8-d460-451e-804f-57af8fe8ca28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:28:45.502603 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b62cb8-d460-451e-804f-57af8fe8ca28 admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:45.502883 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b62cb8-d460-451e-804f-57af8fe8ca28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:45.503163 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b62cb8-d460-451e-804f-57af8fe8ca28 admin admin] PATH_INFO: `/v3/projects/ef95389480e6457987790aad48952c0b/users/b99715cd2bf8446b841725bbd3da2abf/roles/15f634d4a2d247ddab90c458e5e1c141` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:45.505160 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b62cb8-d460-451e-804f-57af8fe8ca28 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef95389480e6457987790aad48952c0b, user_id=b99715cd2bf8446b841725bbd3da2abf, role_id=15f634d4a2d247ddab90c458e5e1c141)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:28:45.507934 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b62cb8-d460-451e-804f-57af8fe8ca28 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:28:45.523664 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 912/1830] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 24 09:28:45 2022] PUT /identity/v3/projects/ef95389480e6457987790aad48952c0b/users/b99715cd2bf8446b841725bbd3da2abf/roles/15f634d4a2d247ddab90c458e5e1c141 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:28:45.531293 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00458688-d40d-441c-b257-0b4417fbab2b None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:45.531576 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00458688-d40d-441c-b257-0b4417fbab2b None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:45.531912 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00458688-d40d-441c-b257-0b4417fbab2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:45.559714 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-00458688-d40d-441c-b257-0b4417fbab2b None None] MFA Rules not processed for user `b99715cd2bf8446b841725bbd3da2abf`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:28:45.561490 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-00458688-d40d-441c-b257-0b4417fbab2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:28:45.630569 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 919/1831] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 09:28:45 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:28:45.641101 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b62cb8-d460-451e-804f-57af8fe8ca28 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:28:45.642860 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-91b62cb8-d460-451e-804f-57af8fe8ca28 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:28:45.699825 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b62cb8-d460-451e-804f-57af8fe8ca28 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:28:45.704013 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b97353f-53ce-47d8-8d0e-35ce1699ef63 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c32b83577c46488906fe409a790b24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:28:45.707774 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b97353f-53ce-47d8-8d0e-35ce1699ef63 service cinder] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:45.707774 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b97353f-53ce-47d8-8d0e-35ce1699ef63 service cinder] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:45.707774 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b97353f-53ce-47d8-8d0e-35ce1699ef63 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:45.714597 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b97353f-53ce-47d8-8d0e-35ce1699ef63 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:28:45.716654 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b97353f-53ce-47d8-8d0e-35ce1699ef63 service cinder] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:28:45.726196 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 913/1832] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:28:45 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:28:45.836091 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00458688-d40d-441c-b257-0b4417fbab2b None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:28:45.857011 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00458688-d40d-441c-b257-0b4417fbab2b None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:28:45.859217 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d5880f-045c-4b78-8616-9d15544686ee None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:45.859820 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d5880f-045c-4b78-8616-9d15544686ee None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:45.862066 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d5880f-045c-4b78-8616-9d15544686ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:45.864279 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5b46b1-6f62-431d-a15c-e7b743c0054a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c32b83577c46488906fe409a790b24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:28:45.866747 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5b46b1-6f62-431d-a15c-e7b743c0054a service cinder] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:45.867059 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5b46b1-6f62-431d-a15c-e7b743c0054a service cinder] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:45.867364 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5b46b1-6f62-431d-a15c-e7b743c0054a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:45.918507 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:28:45.961412 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-e0d5880f-045c-4b78-8616-9d15544686ee None None] MFA Rules not processed for user `a01fa696ca6a42a1b1432a4a6f162b1d`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:28:45.962915 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-e0d5880f-045c-4b78-8616-9d15544686ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:28:45.995334 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5b46b1-6f62-431d-a15c-e7b743c0054a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:28:45.995334 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5b46b1-6f62-431d-a15c-e7b743c0054a service cinder] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:28:45.999868 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 920/1833] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:28:45 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:28:46.053952 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 914/1834] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 09:28:45 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 09:28:46.058333 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5b46b1-6f62-431d-a15c-e7b743c0054a service cinder] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:28:46.138748 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5b46b1-6f62-431d-a15c-e7b743c0054a service cinder] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:28:46.144616 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d209abe5-cc72-4afe-b254-05f5c246df57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:28:46.146573 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d209abe5-cc72-4afe-b254-05f5c246df57 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:46.146573 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d209abe5-cc72-4afe-b254-05f5c246df57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:46.146573 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d209abe5-cc72-4afe-b254-05f5c246df57 admin admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:46.146573 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d209abe5-cc72-4afe-b254-05f5c246df57 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:28:46.149064 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d209abe5-cc72-4afe-b254-05f5c246df57 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:28:46.308683 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 921/1835] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 24 09:28:46 2022] POST /identity/v3/projects => generated 373 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:28:46.313165 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d5880f-045c-4b78-8616-9d15544686ee None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:28:46.323935 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d5880f-045c-4b78-8616-9d15544686ee None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:28:46.382510 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f144d4b8-1888-4781-b1b2-8fe06770a382 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:28:46.382974 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f144d4b8-1888-4781-b1b2-8fe06770a382 admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:46.383257 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f144d4b8-1888-4781-b1b2-8fe06770a382 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:46.383680 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f144d4b8-1888-4781-b1b2-8fe06770a382 admin admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:46.384318 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f144d4b8-1888-4781-b1b2-8fe06770a382 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:28:46.386317 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f144d4b8-1888-4781-b1b2-8fe06770a382 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:28:46.483623 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 915/1836] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 24 09:28:46 2022] POST /identity/v3/users => generated 293 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:28:46.485049 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d209abe5-cc72-4afe-b254-05f5c246df57 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:28:46.497046 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d209abe5-cc72-4afe-b254-05f5c246df57 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:28:46.502291 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f407d159-26a4-429a-86d9-799d2b27c3d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:28:46.503048 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f407d159-26a4-429a-86d9-799d2b27c3d8 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:46.503334 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f407d159-26a4-429a-86d9-799d2b27c3d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:46.504065 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f407d159-26a4-429a-86d9-799d2b27c3d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:46.504315 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f407d159-26a4-429a-86d9-799d2b27c3d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:28:46.506033 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f407d159-26a4-429a-86d9-799d2b27c3d8 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:28:46.518150 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 922/1837] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 24 09:28:46 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:28:46.522388 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f144d4b8-1888-4781-b1b2-8fe06770a382 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:28:46.534007 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f144d4b8-1888-4781-b1b2-8fe06770a382 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:28:46.543529 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478e058f-5d27-45a3-ac96-a9f7b8996af8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:28:46.544909 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478e058f-5d27-45a3-ac96-a9f7b8996af8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:46.545325 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478e058f-5d27-45a3-ac96-a9f7b8996af8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:46.545610 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478e058f-5d27-45a3-ac96-a9f7b8996af8 admin admin] PATH_INFO: `/v3/projects/bd35a900b44b4cb28f1562e1e275726c/users/3390875ed30343a393bc15700b5ace57/roles/15f634d4a2d247ddab90c458e5e1c141` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:46.562344 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:28:46.563441 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478e058f-5d27-45a3-ac96-a9f7b8996af8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd35a900b44b4cb28f1562e1e275726c, user_id=3390875ed30343a393bc15700b5ace57, role_id=15f634d4a2d247ddab90c458e5e1c141)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:28:46.566374 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478e058f-5d27-45a3-ac96-a9f7b8996af8 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:28:46.645186 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 916/1838] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 24 09:28:46 2022] PUT /identity/v3/projects/bd35a900b44b4cb28f1562e1e275726c/users/3390875ed30343a393bc15700b5ace57/roles/15f634d4a2d247ddab90c458e5e1c141 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:28:46.649557 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f407d159-26a4-429a-86d9-799d2b27c3d8 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:28:46.660463 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f407d159-26a4-429a-86d9-799d2b27c3d8 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:28:46.715871 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c1adf4-0623-473c-8e13-df2817027192 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:28:46.716979 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c1adf4-0623-473c-8e13-df2817027192 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:46.717264 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c1adf4-0623-473c-8e13-df2817027192 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:46.717539 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c1adf4-0623-473c-8e13-df2817027192 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:46.718204 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c1adf4-0623-473c-8e13-df2817027192 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:28:46.719825 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c1adf4-0623-473c-8e13-df2817027192 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:28:46.733156 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 923/1839] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 24 09:28:46 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:28:46.734819 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478e058f-5d27-45a3-ac96-a9f7b8996af8 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:28:46.750801 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478e058f-5d27-45a3-ac96-a9f7b8996af8 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:28:46.751392 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe40d5e5-f911-43b0-90b4-420f24a6da9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:28:46.751755 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe40d5e5-f911-43b0-90b4-420f24a6da9c admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:46.751974 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe40d5e5-f911-43b0-90b4-420f24a6da9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:46.752251 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe40d5e5-f911-43b0-90b4-420f24a6da9c admin admin] PATH_INFO: `/v3/projects/bd35a900b44b4cb28f1562e1e275726c/users/3390875ed30343a393bc15700b5ace57/roles/15f634d4a2d247ddab90c458e5e1c141` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:46.754479 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe40d5e5-f911-43b0-90b4-420f24a6da9c admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd35a900b44b4cb28f1562e1e275726c, user_id=3390875ed30343a393bc15700b5ace57, role_id=15f634d4a2d247ddab90c458e5e1c141)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:28:46.757309 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe40d5e5-f911-43b0-90b4-420f24a6da9c admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:28:46.776894 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 917/1840] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 24 09:28:46 2022] PUT /identity/v3/projects/bd35a900b44b4cb28f1562e1e275726c/users/3390875ed30343a393bc15700b5ace57/roles/15f634d4a2d247ddab90c458e5e1c141 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 09:28:47.951646 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e911f390-2e1b-4e2d-b1c3-cff14910aa72 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:47.952153 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e911f390-2e1b-4e2d-b1c3-cff14910aa72 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:47.952268 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e911f390-2e1b-4e2d-b1c3-cff14910aa72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:47.982272 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-e911f390-2e1b-4e2d-b1c3-cff14910aa72 None None] MFA Rules not processed for user `3390875ed30343a393bc15700b5ace57`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:28:47.983712 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-e911f390-2e1b-4e2d-b1c3-cff14910aa72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:28:48.013724 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:28:48.064326 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 924/1841] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 09:28:47 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:28:48.077055 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe40d5e5-f911-43b0-90b4-420f24a6da9c admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:28:48.078843 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-fe40d5e5-f911-43b0-90b4-420f24a6da9c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:28:48.140231 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe40d5e5-f911-43b0-90b4-420f24a6da9c admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:28:48.144042 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343784c6-3985-4183-b3ab-1fd0113ca226 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c32b83577c46488906fe409a790b24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:28:48.144659 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343784c6-3985-4183-b3ab-1fd0113ca226 service cinder] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:48.144942 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343784c6-3985-4183-b3ab-1fd0113ca226 service cinder] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:48.145386 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343784c6-3985-4183-b3ab-1fd0113ca226 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:48.204224 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343784c6-3985-4183-b3ab-1fd0113ca226 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:28:48.205663 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343784c6-3985-4183-b3ab-1fd0113ca226 service cinder] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:28:48.215762 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 918/1842] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:28:48 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:28:50.469248 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e911f390-2e1b-4e2d-b1c3-cff14910aa72 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:28:50.547825 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e911f390-2e1b-4e2d-b1c3-cff14910aa72 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:28:50.548321 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0923bb-9d19-46c4-8154-24cfe4f67ba0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:28:50.549044 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0923bb-9d19-46c4-8154-24cfe4f67ba0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:50.549389 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0923bb-9d19-46c4-8154-24cfe4f67ba0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:50.549734 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0923bb-9d19-46c4-8154-24cfe4f67ba0 admin admin] PATH_INFO: `/v3/users/e2e5d81ba0694d6b8ea7edbd335ea4ad` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:50.550933 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0923bb-9d19-46c4-8154-24cfe4f67ba0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e2e5d81ba0694d6b8ea7edbd335ea4ad)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:28:50.552972 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0923bb-9d19-46c4-8154-24cfe4f67ba0 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:28:50.755811 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-2f0923bb-9d19-46c4-8154-24cfe4f67ba0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e2e5d81ba0694d6b8ea7edbd335ea4ad'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:28:50.756983 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-2f0923bb-9d19-46c4-8154-24cfe4f67ba0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e2e5d81ba0694d6b8ea7edbd335ea4ad'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:28:50.832187 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-2f0923bb-9d19-46c4-8154-24cfe4f67ba0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e2e5d81ba0694d6b8ea7edbd335ea4ad'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:28:50.832802 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-2f0923bb-9d19-46c4-8154-24cfe4f67ba0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e2e5d81ba0694d6b8ea7edbd335ea4ad'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:28:50.840851 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-2f0923bb-9d19-46c4-8154-24cfe4f67ba0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e2e5d81ba0694d6b8ea7edbd335ea4ad'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:28:50.859388 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 925/1843] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 24 09:28:50 2022] DELETE /identity/v3/users/e2e5d81ba0694d6b8ea7edbd335ea4ad => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:28:50.863979 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a310bcc-e0a6-4686-b3a5-e2139c447797 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:50.864487 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a310bcc-e0a6-4686-b3a5-e2139c447797 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:50.864576 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a310bcc-e0a6-4686-b3a5-e2139c447797 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:50.883797 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-9a310bcc-e0a6-4686-b3a5-e2139c447797 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:28:50.885004 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-9a310bcc-e0a6-4686-b3a5-e2139c447797 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:28:50.943669 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 919/1844] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 09:28:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:28:50.953983 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0923bb-9d19-46c4-8154-24cfe4f67ba0 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:28:50.955418 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-2f0923bb-9d19-46c4-8154-24cfe4f67ba0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:28:51.003013 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:28:51.014924 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0923bb-9d19-46c4-8154-24cfe4f67ba0 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:28:51.018657 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfce14c9-4424-4225-82aa-520bc6094541 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e5260e798404e8bb12b25ca8ea97052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:28:51.024641 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfce14c9-4424-4225-82aa-520bc6094541 service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:51.024641 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfce14c9-4424-4225-82aa-520bc6094541 service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:51.024641 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfce14c9-4424-4225-82aa-520bc6094541 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:51.030398 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfce14c9-4424-4225-82aa-520bc6094541 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:28:51.031862 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfce14c9-4424-4225-82aa-520bc6094541 service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:28:51.042059 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 926/1845] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:28:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:28:51.114773 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a310bcc-e0a6-4686-b3a5-e2139c447797 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:28:51.116315 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-9a310bcc-e0a6-4686-b3a5-e2139c447797 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:28:51.152673 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:28:51.180059 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a310bcc-e0a6-4686-b3a5-e2139c447797 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:28:51.185143 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d977868f-97d9-4e2d-a819-ea08b11890d2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c32b83577c46488906fe409a790b24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:28:51.185143 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d977868f-97d9-4e2d-a819-ea08b11890d2 service cinder] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:51.185410 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d977868f-97d9-4e2d-a819-ea08b11890d2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:51.185682 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d977868f-97d9-4e2d-a819-ea08b11890d2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:51.187495 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-d977868f-97d9-4e2d-a819-ea08b11890d2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:28:51.254501 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d977868f-97d9-4e2d-a819-ea08b11890d2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:28:51.263834 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d977868f-97d9-4e2d-a819-ea08b11890d2 service cinder] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:28:51.269751 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 920/1846] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:28:51 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:28:51.644471 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfce14c9-4424-4225-82aa-520bc6094541 service neutron] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:28:51.646212 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-dfce14c9-4424-4225-82aa-520bc6094541 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:28:51.668256 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfce14c9-4424-4225-82aa-520bc6094541 service neutron] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:28:51.677515 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a47d394-31a6-4d3a-b961-a1e7c3835339 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:28:51.678614 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a47d394-31a6-4d3a-b961-a1e7c3835339 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:51.678614 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a47d394-31a6-4d3a-b961-a1e7c3835339 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:51.678614 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a47d394-31a6-4d3a-b961-a1e7c3835339 admin admin] PATH_INFO: `/v3/projects/646e6f6795d5490489a8c3b6c1e9d908` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:51.688453 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a47d394-31a6-4d3a-b961-a1e7c3835339 admin admin] RBAC: Authorizing `identity:delete_project(project_id=646e6f6795d5490489a8c3b6c1e9d908)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:28:51.691138 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a47d394-31a6-4d3a-b961-a1e7c3835339 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:28:51.869136 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-6a47d394-31a6-4d3a-b961-a1e7c3835339 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '646e6f6795d5490489a8c3b6c1e9d908', 'request_id': 'req-6a47d394-31a6-4d3a-b961-a1e7c3835339'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:28:51.878218 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-6a47d394-31a6-4d3a-b961-a1e7c3835339 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '646e6f6795d5490489a8c3b6c1e9d908', 'request_id': 'req-6a47d394-31a6-4d3a-b961-a1e7c3835339'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:28:51.886325 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-6a47d394-31a6-4d3a-b961-a1e7c3835339 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '646e6f6795d5490489a8c3b6c1e9d908', 'request_id': 'req-6a47d394-31a6-4d3a-b961-a1e7c3835339'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:28:51.971105 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-6a47d394-31a6-4d3a-b961-a1e7c3835339 admin admin] The token cache is being invalidate because project 646e6f6795d5490489a8c3b6c1e9d908 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81602) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:28:51.971632 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-6a47d394-31a6-4d3a-b961-a1e7c3835339 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:28:51.973648 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-6a47d394-31a6-4d3a-b961-a1e7c3835339 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:28:51.980645 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 927/1847] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 24 09:28:51 2022] DELETE /identity/v3/projects/646e6f6795d5490489a8c3b6c1e9d908 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:28:51.986717 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d977868f-97d9-4e2d-a819-ea08b11890d2 service cinder] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:28:52.004489 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d977868f-97d9-4e2d-a819-ea08b11890d2 service cinder] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:28:52.097497 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac08d6de-3bd2-4ca7-b35a-aa896440684c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:28:52.098440 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac08d6de-3bd2-4ca7-b35a-aa896440684c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:52.098788 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac08d6de-3bd2-4ca7-b35a-aa896440684c admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:52.099134 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac08d6de-3bd2-4ca7-b35a-aa896440684c admin admin] PATH_INFO: `/v3/users/b99715cd2bf8446b841725bbd3da2abf` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:52.100717 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac08d6de-3bd2-4ca7-b35a-aa896440684c admin admin] RBAC: Authorizing `identity:delete_user(user_id=b99715cd2bf8446b841725bbd3da2abf)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:28:52.102713 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac08d6de-3bd2-4ca7-b35a-aa896440684c admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:28:52.259534 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5887bbfe-d706-4b3b-8d87-def7420a611d None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:52.260078 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5887bbfe-d706-4b3b-8d87-def7420a611d None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:52.260583 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5887bbfe-d706-4b3b-8d87-def7420a611d None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:52.262263 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 928/1848] 10.222.0.16 () {54 vars in 921 bytes} [Wed Aug 24 09:28:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 09:28:52.281786 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16ab36d-4179-412a-9c7e-86d1331e626a None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:52.282075 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16ab36d-4179-412a-9c7e-86d1331e626a None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:52.282351 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16ab36d-4179-412a-9c7e-86d1331e626a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:52.305715 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-f16ab36d-4179-412a-9c7e-86d1331e626a None None] MFA Rules not processed for user `1f055f7dc51d43bda9b315ca67f11d58`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:28:52.306902 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-f16ab36d-4179-412a-9c7e-86d1331e626a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:28:52.351509 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-ac08d6de-3bd2-4ca7-b35a-aa896440684c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b99715cd2bf8446b841725bbd3da2abf'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:28:52.354714 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-ac08d6de-3bd2-4ca7-b35a-aa896440684c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b99715cd2bf8446b841725bbd3da2abf'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:28:52.366003 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-ac08d6de-3bd2-4ca7-b35a-aa896440684c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b99715cd2bf8446b841725bbd3da2abf'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:28:52.374281 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-ac08d6de-3bd2-4ca7-b35a-aa896440684c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b99715cd2bf8446b841725bbd3da2abf'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:28:52.378449 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:28:52.400216 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 929/1849] 10.222.0.16 () {58 vars in 1018 bytes} [Wed Aug 24 09:28:52 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 09:28:52.414166 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16ab36d-4179-412a-9c7e-86d1331e626a None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:28:52.415638 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-f16ab36d-4179-412a-9c7e-86d1331e626a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:28:52.430791 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16ab36d-4179-412a-9c7e-86d1331e626a None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:28:52.433737 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-ac08d6de-3bd2-4ca7-b35a-aa896440684c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b99715cd2bf8446b841725bbd3da2abf'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:28:52.436676 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9019a27-8ba8-448c-a758-62fad651929f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f055f7dc51d43bda9b315ca67f11d58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:28:52.437289 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9019a27-8ba8-448c-a758-62fad651929f service nova] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:52.437564 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9019a27-8ba8-448c-a758-62fad651929f service nova] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:52.437833 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9019a27-8ba8-448c-a758-62fad651929f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:52.440691 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-b9019a27-8ba8-448c-a758-62fad651929f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:28:52.441173 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 921/1850] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 24 09:28:51 2022] DELETE /identity/v3/users/b99715cd2bf8446b841725bbd3da2abf => generated 0 bytes in 455 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:28:52.453982 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9019a27-8ba8-448c-a758-62fad651929f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:28:52.455520 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9019a27-8ba8-448c-a758-62fad651929f service nova] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:28:52.467503 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 930/1851] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:28:52 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:28:52.961956 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac08d6de-3bd2-4ca7-b35a-aa896440684c admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:28:52.963801 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-ac08d6de-3bd2-4ca7-b35a-aa896440684c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:28:52.999909 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9019a27-8ba8-448c-a758-62fad651929f service nova] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:28:53.001435 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-b9019a27-8ba8-448c-a758-62fad651929f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:28:53.027187 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9019a27-8ba8-448c-a758-62fad651929f service nova] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:28:53.061988 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac08d6de-3bd2-4ca7-b35a-aa896440684c admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:28:53.067827 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da387f3-87af-4c6e-be4c-dd32406f2080 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:28:53.068854 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da387f3-87af-4c6e-be4c-dd32406f2080 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:53.069856 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da387f3-87af-4c6e-be4c-dd32406f2080 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:53.070295 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da387f3-87af-4c6e-be4c-dd32406f2080 admin admin] PATH_INFO: `/v3/users/bc655ce402694d6589e848650ce88204` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:53.072857 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da387f3-87af-4c6e-be4c-dd32406f2080 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bc655ce402694d6589e848650ce88204)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:28:53.076813 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da387f3-87af-4c6e-be4c-dd32406f2080 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:28:53.150889 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:28:53.186247 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e920ce65-8157-4dcd-b2c7-ad36b70f14ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:28:53.186916 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e920ce65-8157-4dcd-b2c7-ad36b70f14ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:53.187206 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e920ce65-8157-4dcd-b2c7-ad36b70f14ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:53.187700 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e920ce65-8157-4dcd-b2c7-ad36b70f14ac admin admin] PATH_INFO: `/v3/projects/ef95389480e6457987790aad48952c0b` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:53.196036 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e920ce65-8157-4dcd-b2c7-ad36b70f14ac admin admin] RBAC: Authorizing `identity:delete_project(project_id=ef95389480e6457987790aad48952c0b)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:28:53.198362 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e920ce65-8157-4dcd-b2c7-ad36b70f14ac admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:28:53.346285 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-1da387f3-87af-4c6e-be4c-dd32406f2080 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bc655ce402694d6589e848650ce88204'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:28:53.347037 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-1da387f3-87af-4c6e-be4c-dd32406f2080 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bc655ce402694d6589e848650ce88204'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:28:53.360604 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-1da387f3-87af-4c6e-be4c-dd32406f2080 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bc655ce402694d6589e848650ce88204'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:28:53.369018 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-1da387f3-87af-4c6e-be4c-dd32406f2080 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bc655ce402694d6589e848650ce88204'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:28:53.395763 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-e920ce65-8157-4dcd-b2c7-ad36b70f14ac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ef95389480e6457987790aad48952c0b', 'request_id': 'req-e920ce65-8157-4dcd-b2c7-ad36b70f14ac'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:28:53.399843 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-e920ce65-8157-4dcd-b2c7-ad36b70f14ac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ef95389480e6457987790aad48952c0b', 'request_id': 'req-e920ce65-8157-4dcd-b2c7-ad36b70f14ac'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:28:53.406043 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-e920ce65-8157-4dcd-b2c7-ad36b70f14ac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ef95389480e6457987790aad48952c0b', 'request_id': 'req-e920ce65-8157-4dcd-b2c7-ad36b70f14ac'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:28:53.478264 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-1da387f3-87af-4c6e-be4c-dd32406f2080 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bc655ce402694d6589e848650ce88204'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:28:53.487341 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 922/1852] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 24 09:28:52 2022] DELETE /identity/v3/users/bc655ce402694d6589e848650ce88204 => generated 0 bytes in 524 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:28:53.490053 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3637304a-0e5d-438a-9566-3290229055db None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:53.491035 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3637304a-0e5d-438a-9566-3290229055db None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:53.491323 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3637304a-0e5d-438a-9566-3290229055db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:53.511088 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-3637304a-0e5d-438a-9566-3290229055db None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:28:53.516719 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-3637304a-0e5d-438a-9566-3290229055db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:28:53.570126 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 923/1853] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 09:28:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 09:28:53.582145 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-e920ce65-8157-4dcd-b2c7-ad36b70f14ac admin admin] The token cache is being invalidate because project ef95389480e6457987790aad48952c0b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81602) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:28:53.582145 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-e920ce65-8157-4dcd-b2c7-ad36b70f14ac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:28:53.582145 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-e920ce65-8157-4dcd-b2c7-ad36b70f14ac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:28:53.582145 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 931/1854] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 24 09:28:52 2022] DELETE /identity/v3/projects/ef95389480e6457987790aad48952c0b => generated 0 bytes in 579 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:28:53.582533 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3637304a-0e5d-438a-9566-3290229055db None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:28:53.582533 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-3637304a-0e5d-438a-9566-3290229055db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:28:53.584018 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db063ce-c631-49a1-8092-225aaf8cb7a6 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:53.584307 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db063ce-c631-49a1-8092-225aaf8cb7a6 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:53.584587 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db063ce-c631-49a1-8092-225aaf8cb7a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:53.601213 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:28:53.608713 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-3db063ce-c631-49a1-8092-225aaf8cb7a6 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:28:53.608713 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-3db063ce-c631-49a1-8092-225aaf8cb7a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:28:53.616250 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 932/1855] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 09:28:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:28:53.623870 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b79bd0b-8227-4898-bff6-8668665ed521 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:53.624234 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b79bd0b-8227-4898-bff6-8668665ed521 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:53.624584 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b79bd0b-8227-4898-bff6-8668665ed521 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:53.645149 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-7b79bd0b-8227-4898-bff6-8668665ed521 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:28:53.646349 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-7b79bd0b-8227-4898-bff6-8668665ed521 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:28:53.653947 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3637304a-0e5d-438a-9566-3290229055db None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:28:53.655488 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 933/1856] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 09:28:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 09:28:53.657642 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc76dbd-c8d7-4f8d-a6b9-cfb799f1bebc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e5260e798404e8bb12b25ca8ea97052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:28:53.658254 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc76dbd-c8d7-4f8d-a6b9-cfb799f1bebc service neutron] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:53.658531 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc76dbd-c8d7-4f8d-a6b9-cfb799f1bebc service neutron] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:53.658829 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc76dbd-c8d7-4f8d-a6b9-cfb799f1bebc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:53.660538 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b79bd0b-8227-4898-bff6-8668665ed521 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:28:53.660836 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-8cc76dbd-c8d7-4f8d-a6b9-cfb799f1bebc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:28:53.672779 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc76dbd-c8d7-4f8d-a6b9-cfb799f1bebc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:28:53.673059 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b79bd0b-8227-4898-bff6-8668665ed521 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:28:53.674557 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc76dbd-c8d7-4f8d-a6b9-cfb799f1bebc service neutron] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:28:53.678864 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e408b130-f039-42cd-90e2-fe7f87676c06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:28:53.679468 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e408b130-f039-42cd-90e2-fe7f87676c06 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:53.679780 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e408b130-f039-42cd-90e2-fe7f87676c06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:53.680055 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e408b130-f039-42cd-90e2-fe7f87676c06 admin admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:53.680702 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e408b130-f039-42cd-90e2-fe7f87676c06 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:28:53.681007 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e408b130-f039-42cd-90e2-fe7f87676c06 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:28:53.682661 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e408b130-f039-42cd-90e2-fe7f87676c06 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:28:53.684121 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 924/1857] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:28:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:28:53.698459 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 934/1858] 10.222.0.16 () {58 vars in 1153 bytes} [Wed Aug 24 09:28:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:28:53.701474 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc76dbd-c8d7-4f8d-a6b9-cfb799f1bebc service neutron] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:28:53.726832 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc76dbd-c8d7-4f8d-a6b9-cfb799f1bebc service neutron] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:28:53.744888 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2d20f9-5090-43be-92b8-565bf380c483 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:28:53.745961 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2d20f9-5090-43be-92b8-565bf380c483 admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:53.745961 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2d20f9-5090-43be-92b8-565bf380c483 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:53.746347 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2d20f9-5090-43be-92b8-565bf380c483 admin admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:53.747695 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2d20f9-5090-43be-92b8-565bf380c483 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:28:53.749792 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2d20f9-5090-43be-92b8-565bf380c483 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:28:53.860728 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 925/1859] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 24 09:28:53 2022] POST /identity/v3/projects => generated 381 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 09:28:53.865081 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e408b130-f039-42cd-90e2-fe7f87676c06 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:28:53.877985 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e408b130-f039-42cd-90e2-fe7f87676c06 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:28:53.939718 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62ca81a-afd0-4fcf-b55f-66de658ee3b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:28:53.940340 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62ca81a-afd0-4fcf-b55f-66de658ee3b2 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:53.940737 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62ca81a-afd0-4fcf-b55f-66de658ee3b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:53.940901 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62ca81a-afd0-4fcf-b55f-66de658ee3b2 admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:53.941525 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62ca81a-afd0-4fcf-b55f-66de658ee3b2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:28:53.943471 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62ca81a-afd0-4fcf-b55f-66de658ee3b2 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:28:54.050705 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 935/1860] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 24 09:28:53 2022] POST /identity/v3/users => generated 291 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 09:28:54.055098 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2d20f9-5090-43be-92b8-565bf380c483 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:28:54.065753 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2d20f9-5090-43be-92b8-565bf380c483 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:28:54.072264 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a2dbb3-929c-4178-8c75-b22015503b6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:28:54.073519 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a2dbb3-929c-4178-8c75-b22015503b6a admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:54.073519 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a2dbb3-929c-4178-8c75-b22015503b6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:54.073728 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a2dbb3-929c-4178-8c75-b22015503b6a admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:54.074590 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a2dbb3-929c-4178-8c75-b22015503b6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:28:54.076934 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a2dbb3-929c-4178-8c75-b22015503b6a admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:28:54.089495 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 926/1861] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 24 09:28:54 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 09:28:54.094705 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62ca81a-afd0-4fcf-b55f-66de658ee3b2 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:28:54.106250 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62ca81a-afd0-4fcf-b55f-66de658ee3b2 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:28:54.111088 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58476685-cfe3-4257-8dec-126786701474 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:28:54.111856 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58476685-cfe3-4257-8dec-126786701474 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:54.112212 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58476685-cfe3-4257-8dec-126786701474 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:54.112717 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58476685-cfe3-4257-8dec-126786701474 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:54.113568 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58476685-cfe3-4257-8dec-126786701474 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:28:54.115647 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58476685-cfe3-4257-8dec-126786701474 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:28:54.133095 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 936/1862] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 24 09:28:54 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:28:54.137186 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a2dbb3-929c-4178-8c75-b22015503b6a admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:28:54.147832 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a2dbb3-929c-4178-8c75-b22015503b6a admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:28:54.152596 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e1adb5-9ca0-4c42-a456-3d5b30149fc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:28:54.153869 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e1adb5-9ca0-4c42-a456-3d5b30149fc4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:54.154360 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e1adb5-9ca0-4c42-a456-3d5b30149fc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:54.154730 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e1adb5-9ca0-4c42-a456-3d5b30149fc4 admin admin] PATH_INFO: `/v3/projects/832c1234c07143d2a2450788ecc8c713/users/b7916e0f048c458d860159b6b9cafdaf/roles/136cdf78e00a426997dd4e37faff489b` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:54.170333 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e1adb5-9ca0-4c42-a456-3d5b30149fc4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=832c1234c07143d2a2450788ecc8c713, user_id=b7916e0f048c458d860159b6b9cafdaf, role_id=136cdf78e00a426997dd4e37faff489b)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:28:54.173253 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e1adb5-9ca0-4c42-a456-3d5b30149fc4 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:28:54.268996 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 927/1863] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 24 09:28:54 2022] PUT /identity/v3/projects/832c1234c07143d2a2450788ecc8c713/users/b7916e0f048c458d860159b6b9cafdaf/roles/136cdf78e00a426997dd4e37faff489b => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 09:28:54.283343 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0cfb70-21cb-47db-a7b0-d7f918994af4 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:54.283343 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0cfb70-21cb-47db-a7b0-d7f918994af4 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:54.283343 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0cfb70-21cb-47db-a7b0-d7f918994af4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:54.304817 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e1adb5-9ca0-4c42-a456-3d5b30149fc4 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:28:54.306643 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-20e1adb5-9ca0-4c42-a456-3d5b30149fc4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:28:54.327112 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-9f0cfb70-21cb-47db-a7b0-d7f918994af4 None None] MFA Rules not processed for user `b7916e0f048c458d860159b6b9cafdaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:28:54.332345 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-9f0cfb70-21cb-47db-a7b0-d7f918994af4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:28:54.361088 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:28:54.437833 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 937/1864] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 09:28:54 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:28:54.442215 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0cfb70-21cb-47db-a7b0-d7f918994af4 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:28:54.462700 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e1adb5-9ca0-4c42-a456-3d5b30149fc4 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:28:54.483315 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b69296d-1884-4a9f-92e7-1056a1f22d53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:28:54.485372 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b69296d-1884-4a9f-92e7-1056a1f22d53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:54.485937 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b69296d-1884-4a9f-92e7-1056a1f22d53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:54.486218 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b69296d-1884-4a9f-92e7-1056a1f22d53 admin admin] PATH_INFO: `/v3/projects/b833ebb3692e4b51aceaba3d9da870bf` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:54.488947 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b69296d-1884-4a9f-92e7-1056a1f22d53 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b833ebb3692e4b51aceaba3d9da870bf)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:28:54.490870 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b69296d-1884-4a9f-92e7-1056a1f22d53 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:28:54.531674 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0cfb70-21cb-47db-a7b0-d7f918994af4 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:28:54.538557 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4461f39d-fe76-47f6-b842-3e1671b6172a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:28:54.539510 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4461f39d-fe76-47f6-b842-3e1671b6172a admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:54.539875 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4461f39d-fe76-47f6-b842-3e1671b6172a admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:54.540151 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4461f39d-fe76-47f6-b842-3e1671b6172a admin admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:54.540786 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4461f39d-fe76-47f6-b842-3e1671b6172a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:28:54.542690 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4461f39d-fe76-47f6-b842-3e1671b6172a admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:28:54.624880 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-9b69296d-1884-4a9f-92e7-1056a1f22d53 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b833ebb3692e4b51aceaba3d9da870bf', 'request_id': 'req-9b69296d-1884-4a9f-92e7-1056a1f22d53'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:28:54.638167 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-9b69296d-1884-4a9f-92e7-1056a1f22d53 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b833ebb3692e4b51aceaba3d9da870bf', 'request_id': 'req-9b69296d-1884-4a9f-92e7-1056a1f22d53'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:28:54.643404 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 938/1865] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 24 09:28:54 2022] POST /identity/v3/projects => generated 381 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:28:54.644579 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-9b69296d-1884-4a9f-92e7-1056a1f22d53 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b833ebb3692e4b51aceaba3d9da870bf', 'request_id': 'req-9b69296d-1884-4a9f-92e7-1056a1f22d53'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:28:54.647111 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4461f39d-fe76-47f6-b842-3e1671b6172a admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:28:54.655673 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=92,keystone:INSERT=2,keystone:DELETE=6 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:28:54.700632 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-9b69296d-1884-4a9f-92e7-1056a1f22d53 admin admin] The token cache is being invalidate because project b833ebb3692e4b51aceaba3d9da870bf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:28:54.700949 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-9b69296d-1884-4a9f-92e7-1056a1f22d53 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:28:54.701616 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-9b69296d-1884-4a9f-92e7-1056a1f22d53 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:28:54.703158 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 928/1866] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 24 09:28:54 2022] DELETE /identity/v3/projects/b833ebb3692e4b51aceaba3d9da870bf => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:28:54.706977 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b69296d-1884-4a9f-92e7-1056a1f22d53 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:28:54.708287 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-9b69296d-1884-4a9f-92e7-1056a1f22d53 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:28:54.715000 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4461f39d-fe76-47f6-b842-3e1671b6172a admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:28:54.719732 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0beb1fe8-a9dd-4e63-9306-b52f211ee417 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:28:54.720336 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0beb1fe8-a9dd-4e63-9306-b52f211ee417 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:54.720616 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0beb1fe8-a9dd-4e63-9306-b52f211ee417 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:54.720886 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0beb1fe8-a9dd-4e63-9306-b52f211ee417 admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:54.721503 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0beb1fe8-a9dd-4e63-9306-b52f211ee417 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:28:54.723441 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0beb1fe8-a9dd-4e63-9306-b52f211ee417 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:28:54.773756 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b69296d-1884-4a9f-92e7-1056a1f22d53 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:28:54.778601 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c7f6fa-40d1-4b26-ae16-e8dd204a836a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:28:54.779239 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c7f6fa-40d1-4b26-ae16-e8dd204a836a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:54.779572 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c7f6fa-40d1-4b26-ae16-e8dd204a836a admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:54.779888 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c7f6fa-40d1-4b26-ae16-e8dd204a836a admin admin] PATH_INFO: `/v3/users/f3f6322ca393415f9cc387c822244ca9` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:54.781021 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c7f6fa-40d1-4b26-ae16-e8dd204a836a admin admin] RBAC: Authorizing `identity:delete_user(user_id=f3f6322ca393415f9cc387c822244ca9)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:28:54.782947 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c7f6fa-40d1-4b26-ae16-e8dd204a836a admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:28:54.814426 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 939/1867] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 24 09:28:54 2022] POST /identity/v3/users => generated 297 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 09:28:54.814426 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0beb1fe8-a9dd-4e63-9306-b52f211ee417 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:28:54.814426 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-0beb1fe8-a9dd-4e63-9306-b52f211ee417 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:28:54.827650 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0beb1fe8-a9dd-4e63-9306-b52f211ee417 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:28:54.832352 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e27a91-b936-40b7-9482-0a2dcd60a9a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:28:54.833076 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e27a91-b936-40b7-9482-0a2dcd60a9a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:54.833387 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e27a91-b936-40b7-9482-0a2dcd60a9a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:54.833665 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e27a91-b936-40b7-9482-0a2dcd60a9a8 admin admin] PATH_INFO: `/v3/users/cb4f9bad649d44a9aa6687f00116f9b3` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:54.834826 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e27a91-b936-40b7-9482-0a2dcd60a9a8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cb4f9bad649d44a9aa6687f00116f9b3)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:28:54.840464 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e27a91-b936-40b7-9482-0a2dcd60a9a8 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:28:55.008493 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-f5c7f6fa-40d1-4b26-ae16-e8dd204a836a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f3f6322ca393415f9cc387c822244ca9'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:28:55.009158 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-f5c7f6fa-40d1-4b26-ae16-e8dd204a836a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f3f6322ca393415f9cc387c822244ca9'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:28:55.021295 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-f5c7f6fa-40d1-4b26-ae16-e8dd204a836a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f3f6322ca393415f9cc387c822244ca9'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:28:55.028660 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-f5c7f6fa-40d1-4b26-ae16-e8dd204a836a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f3f6322ca393415f9cc387c822244ca9'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:28:55.081749 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-f6e27a91-b936-40b7-9482-0a2dcd60a9a8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cb4f9bad649d44a9aa6687f00116f9b3'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:28:55.082666 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-f6e27a91-b936-40b7-9482-0a2dcd60a9a8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cb4f9bad649d44a9aa6687f00116f9b3'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:28:55.107569 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-f5c7f6fa-40d1-4b26-ae16-e8dd204a836a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f3f6322ca393415f9cc387c822244ca9'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:28:55.114079 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 929/1868] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 24 09:28:54 2022] DELETE /identity/v3/users/f3f6322ca393415f9cc387c822244ca9 => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 09:28:55.114854 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c7f6fa-40d1-4b26-ae16-e8dd204a836a admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:28:55.117563 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-f5c7f6fa-40d1-4b26-ae16-e8dd204a836a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:28:55.180876 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c7f6fa-40d1-4b26-ae16-e8dd204a836a admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:28:55.186396 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ce1355-c85b-4f83-a3d6-25e12efdea92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:28:55.187166 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ce1355-c85b-4f83-a3d6-25e12efdea92 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:55.187572 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ce1355-c85b-4f83-a3d6-25e12efdea92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:55.187967 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ce1355-c85b-4f83-a3d6-25e12efdea92 admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:55.188615 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ce1355-c85b-4f83-a3d6-25e12efdea92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:28:55.190308 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ce1355-c85b-4f83-a3d6-25e12efdea92 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:28:55.193192 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-f6e27a91-b936-40b7-9482-0a2dcd60a9a8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cb4f9bad649d44a9aa6687f00116f9b3'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:28:55.193911 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-f6e27a91-b936-40b7-9482-0a2dcd60a9a8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cb4f9bad649d44a9aa6687f00116f9b3'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:28:55.200581 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 930/1869] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 24 09:28:55 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:28:55.202765 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-f6e27a91-b936-40b7-9482-0a2dcd60a9a8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cb4f9bad649d44a9aa6687f00116f9b3'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:28:55.204670 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ce1355-c85b-4f83-a3d6-25e12efdea92 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:28:55.207997 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-b5ce1355-c85b-4f83-a3d6-25e12efdea92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:28:55.208353 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=91,keystone:INSERT=4,keystone:DELETE=5 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:28:55.219784 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ce1355-c85b-4f83-a3d6-25e12efdea92 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:28:55.224373 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f503528f-3228-4761-97b5-2234ce77fdbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:28:55.225020 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f503528f-3228-4761-97b5-2234ce77fdbd admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:55.225480 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f503528f-3228-4761-97b5-2234ce77fdbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:55.225753 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f503528f-3228-4761-97b5-2234ce77fdbd admin admin] PATH_INFO: `/v3/projects/972df1958193447e835cb13662304cc1/users/96d49b0ec6154aa3af43042febfae2cf/roles/15f634d4a2d247ddab90c458e5e1c141` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:55.226770 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 940/1870] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 24 09:28:54 2022] DELETE /identity/v3/users/cb4f9bad649d44a9aa6687f00116f9b3 => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:28:55.231094 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd225f3-b295-4d67-a6f0-9a8678e96831 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:55.236548 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd225f3-b295-4d67-a6f0-9a8678e96831 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:55.236548 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd225f3-b295-4d67-a6f0-9a8678e96831 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:55.240381 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f503528f-3228-4761-97b5-2234ce77fdbd admin admin] RBAC: Authorizing `identity:create_grant(project_id=972df1958193447e835cb13662304cc1, user_id=96d49b0ec6154aa3af43042febfae2cf, role_id=15f634d4a2d247ddab90c458e5e1c141)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:28:55.243017 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f503528f-3228-4761-97b5-2234ce77fdbd admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:28:55.250572 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-edd225f3-b295-4d67-a6f0-9a8678e96831 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:28:55.251811 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-edd225f3-b295-4d67-a6f0-9a8678e96831 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:28:55.307793 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 941/1871] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 09:28:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 09:28:55.318847 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd225f3-b295-4d67-a6f0-9a8678e96831 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:28:55.322622 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 931/1872] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 24 09:28:55 2022] PUT /identity/v3/projects/972df1958193447e835cb13662304cc1/users/96d49b0ec6154aa3af43042febfae2cf/roles/15f634d4a2d247ddab90c458e5e1c141 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 09:28:55.322728 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-edd225f3-b295-4d67-a6f0-9a8678e96831 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:28:55.325751 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f503528f-3228-4761-97b5-2234ce77fdbd admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:28:55.392036 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd225f3-b295-4d67-a6f0-9a8678e96831 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:28:55.403682 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f503528f-3228-4761-97b5-2234ce77fdbd admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:28:55.412267 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310f4960-9676-4901-9949-1cc1a69199db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:28:55.413082 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310f4960-9676-4901-9949-1cc1a69199db admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:55.413299 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310f4960-9676-4901-9949-1cc1a69199db admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:55.413713 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310f4960-9676-4901-9949-1cc1a69199db admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:55.414527 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310f4960-9676-4901-9949-1cc1a69199db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:28:55.416447 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310f4960-9676-4901-9949-1cc1a69199db admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:28:55.437225 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 932/1873] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 24 09:28:55 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:28:55.440954 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310f4960-9676-4901-9949-1cc1a69199db admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:28:55.450260 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bd962e-71d5-4ec2-be31-8826342fc5b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e5260e798404e8bb12b25ca8ea97052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:28:55.451077 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bd962e-71d5-4ec2-be31-8826342fc5b6 service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:55.451516 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bd962e-71d5-4ec2-be31-8826342fc5b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:55.451712 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:INSERT=2,keystone:DELETE=11,keystone:SELECT=87 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:28:55.452002 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bd962e-71d5-4ec2-be31-8826342fc5b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:55.457611 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310f4960-9676-4901-9949-1cc1a69199db admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:28:55.463707 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc1e0e1-8e63-4324-b0a1-6070b48cfcf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:28:55.464485 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc1e0e1-8e63-4324-b0a1-6070b48cfcf9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:55.464858 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc1e0e1-8e63-4324-b0a1-6070b48cfcf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:55.466025 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc1e0e1-8e63-4324-b0a1-6070b48cfcf9 admin admin] PATH_INFO: `/v3/projects/972df1958193447e835cb13662304cc1/users/96d49b0ec6154aa3af43042febfae2cf/roles/15f634d4a2d247ddab90c458e5e1c141` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:55.467924 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bd962e-71d5-4ec2-be31-8826342fc5b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:28:55.468281 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc1e0e1-8e63-4324-b0a1-6070b48cfcf9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=972df1958193447e835cb13662304cc1, user_id=96d49b0ec6154aa3af43042febfae2cf, role_id=15f634d4a2d247ddab90c458e5e1c141)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:28:55.470433 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bd962e-71d5-4ec2-be31-8826342fc5b6 service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:28:55.471141 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc1e0e1-8e63-4324-b0a1-6070b48cfcf9 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:28:55.485546 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 942/1874] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:28:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 09:28:55.491628 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 933/1875] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 24 09:28:55 2022] PUT /identity/v3/projects/972df1958193447e835cb13662304cc1/users/96d49b0ec6154aa3af43042febfae2cf/roles/15f634d4a2d247ddab90c458e5e1c141 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:28:55.501300 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb44912d-3e14-400c-8d46-e05a8aac27f5 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:55.501584 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb44912d-3e14-400c-8d46-e05a8aac27f5 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:55.501856 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb44912d-3e14-400c-8d46-e05a8aac27f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:55.534138 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-cb44912d-3e14-400c-8d46-e05a8aac27f5 None None] MFA Rules not processed for user `96d49b0ec6154aa3af43042febfae2cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:28:55.535205 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-cb44912d-3e14-400c-8d46-e05a8aac27f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:28:55.622699 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 943/1876] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 09:28:55 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:28:55.629789 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc1e0e1-8e63-4324-b0a1-6070b48cfcf9 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:28:55.631793 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-6fc1e0e1-8e63-4324-b0a1-6070b48cfcf9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:28:55.641350 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb44912d-3e14-400c-8d46-e05a8aac27f5 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:28:55.643351 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-cb44912d-3e14-400c-8d46-e05a8aac27f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:28:55.725896 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc1e0e1-8e63-4324-b0a1-6070b48cfcf9 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:28:55.730014 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6696081-e427-4baa-a7f3-6aacc1bdcd13 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c32b83577c46488906fe409a790b24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:28:55.731251 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6696081-e427-4baa-a7f3-6aacc1bdcd13 service cinder] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:55.732317 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6696081-e427-4baa-a7f3-6aacc1bdcd13 service cinder] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:55.732786 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6696081-e427-4baa-a7f3-6aacc1bdcd13 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:55.745680 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6696081-e427-4baa-a7f3-6aacc1bdcd13 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:28:55.748375 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb44912d-3e14-400c-8d46-e05a8aac27f5 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:28:55.748499 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6696081-e427-4baa-a7f3-6aacc1bdcd13 service cinder] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:28:55.755638 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f67f51b-9f3c-442a-8cc5-2c6741a24b5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:28:55.757263 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f67f51b-9f3c-442a-8cc5-2c6741a24b5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:55.757263 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f67f51b-9f3c-442a-8cc5-2c6741a24b5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:55.757263 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f67f51b-9f3c-442a-8cc5-2c6741a24b5a admin admin] PATH_INFO: `/v3/projects/ad74b940904840ea9be96637bdcd8660` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:55.758285 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f67f51b-9f3c-442a-8cc5-2c6741a24b5a admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad74b940904840ea9be96637bdcd8660)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:28:55.761069 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f67f51b-9f3c-442a-8cc5-2c6741a24b5a admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:28:55.762359 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 934/1877] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:28:55 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:28:55.807615 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:28:55.933228 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-9f67f51b-9f3c-442a-8cc5-2c6741a24b5a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad74b940904840ea9be96637bdcd8660', 'request_id': 'req-9f67f51b-9f3c-442a-8cc5-2c6741a24b5a'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:28:55.949098 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-9f67f51b-9f3c-442a-8cc5-2c6741a24b5a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad74b940904840ea9be96637bdcd8660', 'request_id': 'req-9f67f51b-9f3c-442a-8cc5-2c6741a24b5a'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:28:55.952908 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6696081-e427-4baa-a7f3-6aacc1bdcd13 service cinder] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:28:55.954494 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-e6696081-e427-4baa-a7f3-6aacc1bdcd13 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:28:55.973325 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6696081-e427-4baa-a7f3-6aacc1bdcd13 service cinder] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:28:55.974998 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-9f67f51b-9f3c-442a-8cc5-2c6741a24b5a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad74b940904840ea9be96637bdcd8660', 'request_id': 'req-9f67f51b-9f3c-442a-8cc5-2c6741a24b5a'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:28:56.024751 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a92d7a5-6876-4367-ae80-2c4ac3b658cf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c32b83577c46488906fe409a790b24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:28:56.026514 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a92d7a5-6876-4367-ae80-2c4ac3b658cf service cinder] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:56.026514 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a92d7a5-6876-4367-ae80-2c4ac3b658cf service cinder] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:56.026514 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a92d7a5-6876-4367-ae80-2c4ac3b658cf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:56.027689 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-4a92d7a5-6876-4367-ae80-2c4ac3b658cf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:28:56.068348 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-9f67f51b-9f3c-442a-8cc5-2c6741a24b5a admin admin] The token cache is being invalidate because project ad74b940904840ea9be96637bdcd8660 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81602) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:28:56.068874 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-9f67f51b-9f3c-442a-8cc5-2c6741a24b5a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:28:56.069620 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-9f67f51b-9f3c-442a-8cc5-2c6741a24b5a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:28:56.071402 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 944/1878] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 24 09:28:55 2022] DELETE /identity/v3/projects/ad74b940904840ea9be96637bdcd8660 => generated 0 bytes in 431 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:28:56.072214 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f67f51b-9f3c-442a-8cc5-2c6741a24b5a admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:28:56.073670 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-9f67f51b-9f3c-442a-8cc5-2c6741a24b5a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:28:56.159219 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a92d7a5-6876-4367-ae80-2c4ac3b658cf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:28:56.160855 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a92d7a5-6876-4367-ae80-2c4ac3b658cf service cinder] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:28:56.164113 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f67f51b-9f3c-442a-8cc5-2c6741a24b5a admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:28:56.173900 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 935/1879] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:28:55 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 09:28:56.173900 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b8de74-6d32-4c87-8972-ebf6367abaff None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:56.173900 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b8de74-6d32-4c87-8972-ebf6367abaff None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:56.173900 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b8de74-6d32-4c87-8972-ebf6367abaff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:56.185333 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613c03ff-32b4-4c68-bb3f-b40730cc164f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:28:56.188688 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613c03ff-32b4-4c68-bb3f-b40730cc164f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:56.189196 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613c03ff-32b4-4c68-bb3f-b40730cc164f admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:56.189671 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613c03ff-32b4-4c68-bb3f-b40730cc164f admin admin] PATH_INFO: `/v3/projects/820e170187a442a99dc2e348cf0fc4bc` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:56.193257 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613c03ff-32b4-4c68-bb3f-b40730cc164f admin admin] RBAC: Authorizing `identity:delete_project(project_id=820e170187a442a99dc2e348cf0fc4bc)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:28:56.198395 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613c03ff-32b4-4c68-bb3f-b40730cc164f admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:28:56.203496 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-13b8de74-6d32-4c87-8972-ebf6367abaff None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:28:56.205180 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-13b8de74-6d32-4c87-8972-ebf6367abaff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:28:56.217001 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 936/1880] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 09:28:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:28:56.231899 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c226dcc-3924-48d1-80ae-8be93aefeabc None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:56.238227 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c226dcc-3924-48d1-80ae-8be93aefeabc None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:56.238227 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c226dcc-3924-48d1-80ae-8be93aefeabc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:56.262024 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-1c226dcc-3924-48d1-80ae-8be93aefeabc None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:28:56.264184 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-1c226dcc-3924-48d1-80ae-8be93aefeabc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:28:56.272868 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 937/1881] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 09:28:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:28:56.278427 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c226dcc-3924-48d1-80ae-8be93aefeabc None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:28:56.294308 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c226dcc-3924-48d1-80ae-8be93aefeabc None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:28:56.307295 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a8ce09-8651-4fe9-a59e-046d2468af79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:28:56.307994 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a8ce09-8651-4fe9-a59e-046d2468af79 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:56.308274 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a8ce09-8651-4fe9-a59e-046d2468af79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:56.308643 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a8ce09-8651-4fe9-a59e-046d2468af79 admin admin] PATH_INFO: `/v3/domains` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:56.309401 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a8ce09-8651-4fe9-a59e-046d2468af79 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:28:56.309658 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a8ce09-8651-4fe9-a59e-046d2468af79 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:28:56.311390 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a8ce09-8651-4fe9-a59e-046d2468af79 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:28:56.358877 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 938/1882] 10.222.0.16 () {58 vars in 1153 bytes} [Wed Aug 24 09:28:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:28:56.363231 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a8ce09-8651-4fe9-a59e-046d2468af79 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:28:56.372556 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-613c03ff-32b4-4c68-bb3f-b40730cc164f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '820e170187a442a99dc2e348cf0fc4bc', 'request_id': 'req-613c03ff-32b4-4c68-bb3f-b40730cc164f'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:28:56.375315 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a8ce09-8651-4fe9-a59e-046d2468af79 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:28:56.380335 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-613c03ff-32b4-4c68-bb3f-b40730cc164f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '820e170187a442a99dc2e348cf0fc4bc', 'request_id': 'req-613c03ff-32b4-4c68-bb3f-b40730cc164f'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:28:56.391832 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-613c03ff-32b4-4c68-bb3f-b40730cc164f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '820e170187a442a99dc2e348cf0fc4bc', 'request_id': 'req-613c03ff-32b4-4c68-bb3f-b40730cc164f'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:28:56.493464 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-613c03ff-32b4-4c68-bb3f-b40730cc164f admin admin] The token cache is being invalidate because project 820e170187a442a99dc2e348cf0fc4bc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81602) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:28:56.493759 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-613c03ff-32b4-4c68-bb3f-b40730cc164f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:28:56.494629 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-613c03ff-32b4-4c68-bb3f-b40730cc164f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:28:56.497438 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 945/1883] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 24 09:28:56 2022] DELETE /identity/v3/projects/820e170187a442a99dc2e348cf0fc4bc => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:28:56.501743 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613c03ff-32b4-4c68-bb3f-b40730cc164f admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:28:56.502565 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ac005c-0f25-4b0b-b178-8bfc966f2779 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:28:56.503360 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ac005c-0f25-4b0b-b178-8bfc966f2779 admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:56.503992 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ac005c-0f25-4b0b-b178-8bfc966f2779 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:56.504412 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ac005c-0f25-4b0b-b178-8bfc966f2779 admin admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:56.509108 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ac005c-0f25-4b0b-b178-8bfc966f2779 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:28:56.510024 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-613c03ff-32b4-4c68-bb3f-b40730cc164f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:28:56.520553 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:28:56.523210 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ac005c-0f25-4b0b-b178-8bfc966f2779 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:28:56.535932 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613c03ff-32b4-4c68-bb3f-b40730cc164f admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:28:56.546166 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2842fda-312f-4214-851a-b22a7a72ae47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:28:56.546828 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2842fda-312f-4214-851a-b22a7a72ae47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:56.547108 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2842fda-312f-4214-851a-b22a7a72ae47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:56.547382 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2842fda-312f-4214-851a-b22a7a72ae47 admin admin] PATH_INFO: `/v3/users/1ee501ef4b9e4233a415891ce2c68f3d` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:56.550924 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2842fda-312f-4214-851a-b22a7a72ae47 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1ee501ef4b9e4233a415891ce2c68f3d)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:28:56.553871 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2842fda-312f-4214-851a-b22a7a72ae47 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:28:56.672931 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 939/1884] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 24 09:28:56 2022] POST /identity/v3/projects => generated 371 bytes in 310 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:28:56.676861 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ac005c-0f25-4b0b-b178-8bfc966f2779 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:28:56.680704 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-d2ac005c-0f25-4b0b-b178-8bfc966f2779 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:28:56.788761 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-a2842fda-312f-4214-851a-b22a7a72ae47 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1ee501ef4b9e4233a415891ce2c68f3d'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:28:56.795194 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-a2842fda-312f-4214-851a-b22a7a72ae47 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1ee501ef4b9e4233a415891ce2c68f3d'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:28:56.819304 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ac005c-0f25-4b0b-b178-8bfc966f2779 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:28:56.845432 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66726a74-6e58-4000-a019-b71d1cced26a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:28:56.846109 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66726a74-6e58-4000-a019-b71d1cced26a admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:56.851191 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66726a74-6e58-4000-a019-b71d1cced26a admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:56.851492 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66726a74-6e58-4000-a019-b71d1cced26a admin admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:56.852196 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66726a74-6e58-4000-a019-b71d1cced26a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:28:56.854857 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66726a74-6e58-4000-a019-b71d1cced26a admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:28:56.890073 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-a2842fda-312f-4214-851a-b22a7a72ae47 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1ee501ef4b9e4233a415891ce2c68f3d'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:28:56.890693 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-a2842fda-312f-4214-851a-b22a7a72ae47 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1ee501ef4b9e4233a415891ce2c68f3d'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:28:56.900130 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-a2842fda-312f-4214-851a-b22a7a72ae47 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1ee501ef4b9e4233a415891ce2c68f3d'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:28:56.936717 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 946/1885] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 24 09:28:56 2022] DELETE /identity/v3/users/1ee501ef4b9e4233a415891ce2c68f3d => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:28:56.938053 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2842fda-312f-4214-851a-b22a7a72ae47 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:28:56.939460 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-a2842fda-312f-4214-851a-b22a7a72ae47 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:28:56.953278 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 940/1886] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 24 09:28:56 2022] POST /identity/v3/users => generated 286 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:28:56.965317 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66726a74-6e58-4000-a019-b71d1cced26a admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:28:56.967922 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-66726a74-6e58-4000-a019-b71d1cced26a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:28:56.983032 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66726a74-6e58-4000-a019-b71d1cced26a admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:28:57.034548 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2842fda-312f-4214-851a-b22a7a72ae47 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:28:57.043300 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3435f9ec-726b-4437-9f44-38f17b124de2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:28:57.045327 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3435f9ec-726b-4437-9f44-38f17b124de2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:57.046139 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3435f9ec-726b-4437-9f44-38f17b124de2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:57.046648 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3435f9ec-726b-4437-9f44-38f17b124de2 admin admin] PATH_INFO: `/v3/users/b7916e0f048c458d860159b6b9cafdaf` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:57.049322 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3435f9ec-726b-4437-9f44-38f17b124de2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b7916e0f048c458d860159b6b9cafdaf)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:28:57.052632 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3435f9ec-726b-4437-9f44-38f17b124de2 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:28:57.069011 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2308f804-3970-477e-968c-3628c036048e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:28:57.069697 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2308f804-3970-477e-968c-3628c036048e admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:57.069982 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2308f804-3970-477e-968c-3628c036048e admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:57.070256 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2308f804-3970-477e-968c-3628c036048e admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:57.071242 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2308f804-3970-477e-968c-3628c036048e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:28:57.077699 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2308f804-3970-477e-968c-3628c036048e admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:28:57.093857 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 941/1887] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 24 09:28:56 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 09:28:57.100004 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2308f804-3970-477e-968c-3628c036048e admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:28:57.120117 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2308f804-3970-477e-968c-3628c036048e admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:28:57.126425 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f292462d-a516-45e7-9c34-ea1e8ab290c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:28:57.127909 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f292462d-a516-45e7-9c34-ea1e8ab290c3 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:57.128249 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f292462d-a516-45e7-9c34-ea1e8ab290c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:57.129684 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f292462d-a516-45e7-9c34-ea1e8ab290c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:57.130725 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f292462d-a516-45e7-9c34-ea1e8ab290c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:28:57.132559 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f292462d-a516-45e7-9c34-ea1e8ab290c3 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:28:57.149769 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 942/1888] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 24 09:28:57 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:28:57.151398 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f292462d-a516-45e7-9c34-ea1e8ab290c3 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:28:57.164298 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f292462d-a516-45e7-9c34-ea1e8ab290c3 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:28:57.170055 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07ad039-6c65-41c9-80ea-8736da96ef43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:28:57.174422 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07ad039-6c65-41c9-80ea-8736da96ef43 admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:57.174422 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07ad039-6c65-41c9-80ea-8736da96ef43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:57.174422 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07ad039-6c65-41c9-80ea-8736da96ef43 admin admin] PATH_INFO: `/v3/projects/20dbba5a31cd4d77a878e766a5aab0d0/users/ebef937c678346eabf5acec3622f3f9a/roles/136cdf78e00a426997dd4e37faff489b` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:57.188765 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07ad039-6c65-41c9-80ea-8736da96ef43 admin admin] RBAC: Authorizing `identity:create_grant(project_id=20dbba5a31cd4d77a878e766a5aab0d0, user_id=ebef937c678346eabf5acec3622f3f9a, role_id=136cdf78e00a426997dd4e37faff489b)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:28:57.191638 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07ad039-6c65-41c9-80ea-8736da96ef43 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:28:57.310371 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 943/1889] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 24 09:28:57 2022] PUT /identity/v3/projects/20dbba5a31cd4d77a878e766a5aab0d0/users/ebef937c678346eabf5acec3622f3f9a/roles/136cdf78e00a426997dd4e37faff489b => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:28:57.320526 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a555482e-ff99-4caa-b37a-49fdb0d92fd9 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:57.321269 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a555482e-ff99-4caa-b37a-49fdb0d92fd9 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:57.321879 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a555482e-ff99-4caa-b37a-49fdb0d92fd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:57.381136 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-3435f9ec-726b-4437-9f44-38f17b124de2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b7916e0f048c458d860159b6b9cafdaf'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:28:57.382623 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-3435f9ec-726b-4437-9f44-38f17b124de2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b7916e0f048c458d860159b6b9cafdaf'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:28:57.412185 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-a555482e-ff99-4caa-b37a-49fdb0d92fd9 None None] MFA Rules not processed for user `ebef937c678346eabf5acec3622f3f9a`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:28:57.413495 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-a555482e-ff99-4caa-b37a-49fdb0d92fd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:28:57.438520 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:28:57.467331 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-3435f9ec-726b-4437-9f44-38f17b124de2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b7916e0f048c458d860159b6b9cafdaf'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:28:57.468080 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-3435f9ec-726b-4437-9f44-38f17b124de2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b7916e0f048c458d860159b6b9cafdaf'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:28:57.484801 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-3435f9ec-726b-4437-9f44-38f17b124de2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b7916e0f048c458d860159b6b9cafdaf'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:28:57.519185 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 944/1890] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 09:28:57 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 09:28:57.521714 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 947/1891] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 24 09:28:56 2022] DELETE /identity/v3/users/b7916e0f048c458d860159b6b9cafdaf => generated 0 bytes in 584 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:28:57.530718 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1e90e8-c185-498f-aaf1-74161a02ac6f None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:57.531075 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1e90e8-c185-498f-aaf1-74161a02ac6f None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:57.531634 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1e90e8-c185-498f-aaf1-74161a02ac6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:57.544831 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3435f9ec-726b-4437-9f44-38f17b124de2 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:28:57.546345 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-3435f9ec-726b-4437-9f44-38f17b124de2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:28:57.577648 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-bb1e90e8-c185-498f-aaf1-74161a02ac6f None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:28:57.579042 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-bb1e90e8-c185-498f-aaf1-74161a02ac6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:28:57.654048 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3435f9ec-726b-4437-9f44-38f17b124de2 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:28:57.667040 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687ee1bd-18a9-4b7e-9f1d-975512cfc814 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c32b83577c46488906fe409a790b24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:28:57.667828 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687ee1bd-18a9-4b7e-9f1d-975512cfc814 service cinder] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:57.668317 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687ee1bd-18a9-4b7e-9f1d-975512cfc814 service cinder] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:57.669241 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687ee1bd-18a9-4b7e-9f1d-975512cfc814 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:57.678803 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:28:57.708566 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 945/1892] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 09:28:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:28:57.729359 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1e90e8-c185-498f-aaf1-74161a02ac6f None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:28:57.730923 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-bb1e90e8-c185-498f-aaf1-74161a02ac6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:28:57.792784 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687ee1bd-18a9-4b7e-9f1d-975512cfc814 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:28:57.795009 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687ee1bd-18a9-4b7e-9f1d-975512cfc814 service cinder] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:28:57.821372 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 948/1893] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:28:57 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 277 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 09:28:57.840150 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687ee1bd-18a9-4b7e-9f1d-975512cfc814 service cinder] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:28:57.843297 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-687ee1bd-18a9-4b7e-9f1d-975512cfc814 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:28:57.874278 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687ee1bd-18a9-4b7e-9f1d-975512cfc814 service cinder] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:28:57.878466 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1e90e8-c185-498f-aaf1-74161a02ac6f None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:28:57.881276 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1818a73a-059a-47ef-99c0-1aad6096fdf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:28:57.882677 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1818a73a-059a-47ef-99c0-1aad6096fdf2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:57.883054 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1818a73a-059a-47ef-99c0-1aad6096fdf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:57.884297 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1818a73a-059a-47ef-99c0-1aad6096fdf2 admin admin] PATH_INFO: `/v3/users/ebef937c678346eabf5acec3622f3f9a` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:57.886029 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1818a73a-059a-47ef-99c0-1aad6096fdf2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ebef937c678346eabf5acec3622f3f9a)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:28:57.888889 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a506ac-ae98-4a79-a553-dca632922afa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e5260e798404e8bb12b25ca8ea97052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:28:57.890919 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1818a73a-059a-47ef-99c0-1aad6096fdf2 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:28:57.891131 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a506ac-ae98-4a79-a553-dca632922afa service neutron] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:57.891623 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a506ac-ae98-4a79-a553-dca632922afa service neutron] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:57.892039 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a506ac-ae98-4a79-a553-dca632922afa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:57.918730 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a506ac-ae98-4a79-a553-dca632922afa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:28:57.920320 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a506ac-ae98-4a79-a553-dca632922afa service neutron] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:28:57.943571 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 946/1894] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:28:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 09:28:58.121935 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-1818a73a-059a-47ef-99c0-1aad6096fdf2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ebef937c678346eabf5acec3622f3f9a'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:28:58.122620 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-1818a73a-059a-47ef-99c0-1aad6096fdf2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ebef937c678346eabf5acec3622f3f9a'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:28:58.144420 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a506ac-ae98-4a79-a553-dca632922afa service neutron] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:28:58.144420 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-42a506ac-ae98-4a79-a553-dca632922afa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:28:58.155938 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a506ac-ae98-4a79-a553-dca632922afa service neutron] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:28:58.160589 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94fcb5f-eaab-4706-b9b8-1c0624682da9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:28:58.161393 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94fcb5f-eaab-4706-b9b8-1c0624682da9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:58.161828 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94fcb5f-eaab-4706-b9b8-1c0624682da9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:58.162300 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94fcb5f-eaab-4706-b9b8-1c0624682da9 admin admin] PATH_INFO: `/v3/projects/bcb0ac58da8d49559c4c24dff72e54fd` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:58.163476 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94fcb5f-eaab-4706-b9b8-1c0624682da9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bcb0ac58da8d49559c4c24dff72e54fd)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:28:58.165606 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94fcb5f-eaab-4706-b9b8-1c0624682da9 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:28:58.261930 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-1818a73a-059a-47ef-99c0-1aad6096fdf2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ebef937c678346eabf5acec3622f3f9a'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:28:58.263371 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-1818a73a-059a-47ef-99c0-1aad6096fdf2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ebef937c678346eabf5acec3622f3f9a'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:28:58.274363 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-1818a73a-059a-47ef-99c0-1aad6096fdf2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ebef937c678346eabf5acec3622f3f9a'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:28:58.316941 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 949/1895] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 24 09:28:57 2022] DELETE /identity/v3/users/ebef937c678346eabf5acec3622f3f9a => generated 0 bytes in 477 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:28:58.324083 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac759f7-fc1d-440f-b939-5a8c7f6da22f None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:58.327669 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac759f7-fc1d-440f-b939-5a8c7f6da22f None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:58.327669 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac759f7-fc1d-440f-b939-5a8c7f6da22f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:58.332102 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-e94fcb5f-eaab-4706-b9b8-1c0624682da9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bcb0ac58da8d49559c4c24dff72e54fd', 'request_id': 'req-e94fcb5f-eaab-4706-b9b8-1c0624682da9'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:28:58.342594 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-e94fcb5f-eaab-4706-b9b8-1c0624682da9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bcb0ac58da8d49559c4c24dff72e54fd', 'request_id': 'req-e94fcb5f-eaab-4706-b9b8-1c0624682da9'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:28:58.348477 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-e94fcb5f-eaab-4706-b9b8-1c0624682da9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bcb0ac58da8d49559c4c24dff72e54fd', 'request_id': 'req-e94fcb5f-eaab-4706-b9b8-1c0624682da9'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:28:58.359867 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-6ac759f7-fc1d-440f-b939-5a8c7f6da22f None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:28:58.361160 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-6ac759f7-fc1d-440f-b939-5a8c7f6da22f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:28:58.460844 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-e94fcb5f-eaab-4706-b9b8-1c0624682da9 admin admin] The token cache is being invalidate because project bcb0ac58da8d49559c4c24dff72e54fd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:28:58.461104 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-e94fcb5f-eaab-4706-b9b8-1c0624682da9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:28:58.461968 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 950/1896] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 09:28:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:28:58.463327 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac759f7-fc1d-440f-b939-5a8c7f6da22f None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:28:58.465108 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-6ac759f7-fc1d-440f-b939-5a8c7f6da22f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:28:58.465251 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-e94fcb5f-eaab-4706-b9b8-1c0624682da9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:28:58.466879 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 947/1897] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 24 09:28:58 2022] DELETE /identity/v3/projects/bcb0ac58da8d49559c4c24dff72e54fd => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 09:28:58.471761 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82075f7f-486f-4544-bcb5-6d8e29e0a545 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:58.472054 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82075f7f-486f-4544-bcb5-6d8e29e0a545 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:58.472322 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82075f7f-486f-4544-bcb5-6d8e29e0a545 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:58.482890 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac759f7-fc1d-440f-b939-5a8c7f6da22f None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:28:58.487985 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a27b47-83dc-4e04-8e8a-4b6f513e05fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:28:58.488956 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a27b47-83dc-4e04-8e8a-4b6f513e05fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:58.489461 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a27b47-83dc-4e04-8e8a-4b6f513e05fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:58.489913 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a27b47-83dc-4e04-8e8a-4b6f513e05fc admin admin] PATH_INFO: `/v3/projects/832c1234c07143d2a2450788ecc8c713` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:58.491502 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a27b47-83dc-4e04-8e8a-4b6f513e05fc admin admin] RBAC: Authorizing `identity:delete_project(project_id=832c1234c07143d2a2450788ecc8c713)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:28:58.494977 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a27b47-83dc-4e04-8e8a-4b6f513e05fc admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:28:58.509033 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-82075f7f-486f-4544-bcb5-6d8e29e0a545 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:28:58.511034 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-82075f7f-486f-4544-bcb5-6d8e29e0a545 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:28:58.531862 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 948/1898] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 09:28:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 09:28:58.531862 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82075f7f-486f-4544-bcb5-6d8e29e0a545 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:28:58.532377 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-82075f7f-486f-4544-bcb5-6d8e29e0a545 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:28:58.606515 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82075f7f-486f-4544-bcb5-6d8e29e0a545 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:28:58.623529 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-d8a27b47-83dc-4e04-8e8a-4b6f513e05fc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '832c1234c07143d2a2450788ecc8c713', 'request_id': 'req-d8a27b47-83dc-4e04-8e8a-4b6f513e05fc'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:28:58.631531 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-d8a27b47-83dc-4e04-8e8a-4b6f513e05fc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '832c1234c07143d2a2450788ecc8c713', 'request_id': 'req-d8a27b47-83dc-4e04-8e8a-4b6f513e05fc'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:28:58.641286 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-d8a27b47-83dc-4e04-8e8a-4b6f513e05fc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '832c1234c07143d2a2450788ecc8c713', 'request_id': 'req-d8a27b47-83dc-4e04-8e8a-4b6f513e05fc'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:28:58.663840 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0834b293-f378-4383-9863-1323ed784895 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e5260e798404e8bb12b25ca8ea97052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:28:58.664450 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0834b293-f378-4383-9863-1323ed784895 service neutron] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:58.664727 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0834b293-f378-4383-9863-1323ed784895 service neutron] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:58.664999 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0834b293-f378-4383-9863-1323ed784895 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:58.666750 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-0834b293-f378-4383-9863-1323ed784895 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:28:58.683266 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:28:58.729646 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-d8a27b47-83dc-4e04-8e8a-4b6f513e05fc admin admin] The token cache is being invalidate because project 832c1234c07143d2a2450788ecc8c713 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81602) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:28:58.730002 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-d8a27b47-83dc-4e04-8e8a-4b6f513e05fc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:28:58.736108 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-d8a27b47-83dc-4e04-8e8a-4b6f513e05fc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:28:58.736108 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 951/1899] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 24 09:28:58 2022] DELETE /identity/v3/projects/832c1234c07143d2a2450788ecc8c713 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:28:58.736108 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212f3f3d-9ff1-464e-a400-c930bd549939 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:58.736108 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212f3f3d-9ff1-464e-a400-c930bd549939 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:58.736108 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212f3f3d-9ff1-464e-a400-c930bd549939 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:58.752675 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0834b293-f378-4383-9863-1323ed784895 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:28:58.754391 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0834b293-f378-4383-9863-1323ed784895 service neutron] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:28:58.757299 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-212f3f3d-9ff1-464e-a400-c930bd549939 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:28:58.758713 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-212f3f3d-9ff1-464e-a400-c930bd549939 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:28:58.764849 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 949/1900] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:28:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:28:58.767510 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 952/1901] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 09:28:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:28:58.772999 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212f3f3d-9ff1-464e-a400-c930bd549939 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:28:58.773101 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0834b293-f378-4383-9863-1323ed784895 service neutron] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:28:58.776708 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-0834b293-f378-4383-9863-1323ed784895 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:28:58.783374 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212f3f3d-9ff1-464e-a400-c930bd549939 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:28:58.790740 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a51d18-371a-45be-8d54-5398dec08165 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:28:58.792644 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a51d18-371a-45be-8d54-5398dec08165 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:58.792644 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a51d18-371a-45be-8d54-5398dec08165 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:58.792644 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a51d18-371a-45be-8d54-5398dec08165 admin admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:58.796274 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0834b293-f378-4383-9863-1323ed784895 service neutron] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:28:58.800202 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a51d18-371a-45be-8d54-5398dec08165 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:28:58.800806 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a51d18-371a-45be-8d54-5398dec08165 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:28:58.803213 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a51d18-371a-45be-8d54-5398dec08165 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:28:58.803706 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f34a52c-9ecc-49c1-babf-5299656a036d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:28:58.804897 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f34a52c-9ecc-49c1-babf-5299656a036d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:58.805303 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f34a52c-9ecc-49c1-babf-5299656a036d admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:58.805921 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f34a52c-9ecc-49c1-babf-5299656a036d admin admin] PATH_INFO: `/v3/users/96d49b0ec6154aa3af43042febfae2cf` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:58.808263 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f34a52c-9ecc-49c1-babf-5299656a036d admin admin] RBAC: Authorizing `identity:delete_user(user_id=96d49b0ec6154aa3af43042febfae2cf)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:28:58.811469 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f34a52c-9ecc-49c1-babf-5299656a036d admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:28:58.824024 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 953/1902] 10.222.0.16 () {58 vars in 1153 bytes} [Wed Aug 24 09:28:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:28:58.826317 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a51d18-371a-45be-8d54-5398dec08165 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:28:58.841587 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a51d18-371a-45be-8d54-5398dec08165 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:28:58.853521 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca20a100-9e9d-4614-be83-89811b1333f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:28:58.854636 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca20a100-9e9d-4614-be83-89811b1333f7 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:58.855249 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca20a100-9e9d-4614-be83-89811b1333f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:58.855718 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca20a100-9e9d-4614-be83-89811b1333f7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:58.856391 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca20a100-9e9d-4614-be83-89811b1333f7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:28:58.858979 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca20a100-9e9d-4614-be83-89811b1333f7 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:28:58.884150 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:28:58.967909 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 954/1903] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 24 09:28:58 2022] POST /identity/v3/projects => generated 363 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:28:58.972160 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca20a100-9e9d-4614-be83-89811b1333f7 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:28:59.021213 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-6f34a52c-9ecc-49c1-babf-5299656a036d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '96d49b0ec6154aa3af43042febfae2cf'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:28:59.040618 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-6f34a52c-9ecc-49c1-babf-5299656a036d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '96d49b0ec6154aa3af43042febfae2cf'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:28:59.049052 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca20a100-9e9d-4614-be83-89811b1333f7 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:28:59.052078 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-6f34a52c-9ecc-49c1-babf-5299656a036d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '96d49b0ec6154aa3af43042febfae2cf'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:28:59.055606 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438b526e-d7ed-400f-8a56-77f45d1735f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:28:59.058269 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438b526e-d7ed-400f-8a56-77f45d1735f0 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:59.058269 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438b526e-d7ed-400f-8a56-77f45d1735f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:59.058269 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438b526e-d7ed-400f-8a56-77f45d1735f0 admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:59.058269 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438b526e-d7ed-400f-8a56-77f45d1735f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:28:59.059949 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438b526e-d7ed-400f-8a56-77f45d1735f0 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:28:59.060402 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-6f34a52c-9ecc-49c1-babf-5299656a036d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '96d49b0ec6154aa3af43042febfae2cf'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:28:59.146590 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-6f34a52c-9ecc-49c1-babf-5299656a036d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '96d49b0ec6154aa3af43042febfae2cf'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:28:59.156792 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 950/1904] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 24 09:28:58 2022] DELETE /identity/v3/users/96d49b0ec6154aa3af43042febfae2cf => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:28:59.201942 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 955/1905] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 24 09:28:58 2022] POST /identity/v3/users => generated 282 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 09:28:59.203257 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f34a52c-9ecc-49c1-babf-5299656a036d admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:28:59.204940 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-6f34a52c-9ecc-49c1-babf-5299656a036d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:28:59.270410 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f34a52c-9ecc-49c1-babf-5299656a036d admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:28:59.275079 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae85556-6219-45a6-96de-a84d267dbf19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:28:59.275737 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae85556-6219-45a6-96de-a84d267dbf19 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:59.276017 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae85556-6219-45a6-96de-a84d267dbf19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:59.276288 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae85556-6219-45a6-96de-a84d267dbf19 admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:59.276928 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae85556-6219-45a6-96de-a84d267dbf19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:28:59.278567 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae85556-6219-45a6-96de-a84d267dbf19 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:28:59.288141 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 951/1906] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 24 09:28:59 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:28:59.292459 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438b526e-d7ed-400f-8a56-77f45d1735f0 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:28:59.303381 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438b526e-d7ed-400f-8a56-77f45d1735f0 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:28:59.308418 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68f21a4-1957-4843-825e-ed3c67158d91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:28:59.309026 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68f21a4-1957-4843-825e-ed3c67158d91 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:59.309897 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68f21a4-1957-4843-825e-ed3c67158d91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:59.310174 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68f21a4-1957-4843-825e-ed3c67158d91 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:59.310827 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68f21a4-1957-4843-825e-ed3c67158d91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:28:59.312491 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68f21a4-1957-4843-825e-ed3c67158d91 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:28:59.325640 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 956/1907] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 24 09:28:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:28:59.326181 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae85556-6219-45a6-96de-a84d267dbf19 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:28:59.326823 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68f21a4-1957-4843-825e-ed3c67158d91 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:28:59.328125 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-aae85556-6219-45a6-96de-a84d267dbf19 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:28:59.337303 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68f21a4-1957-4843-825e-ed3c67158d91 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:28:59.342252 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f106c6-003f-440c-ab09-6510a44c377a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:28:59.342928 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f106c6-003f-440c-ab09-6510a44c377a admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:59.343207 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f106c6-003f-440c-ab09-6510a44c377a admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:59.343337 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae85556-6219-45a6-96de-a84d267dbf19 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:28:59.343483 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f106c6-003f-440c-ab09-6510a44c377a admin admin] PATH_INFO: `/v3/projects/9dda1c7250c7408593271d4252698e79/users/57d46739f2e548d9b5e7de3f54b64aea/roles/136cdf78e00a426997dd4e37faff489b` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:59.350217 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed987a9-6392-4c03-94cb-83ac258d5397 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:28:59.351303 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed987a9-6392-4c03-94cb-83ac258d5397 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:59.351382 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed987a9-6392-4c03-94cb-83ac258d5397 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:59.351702 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed987a9-6392-4c03-94cb-83ac258d5397 admin admin] PATH_INFO: `/v3/projects/20dbba5a31cd4d77a878e766a5aab0d0` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:59.352838 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed987a9-6392-4c03-94cb-83ac258d5397 admin admin] RBAC: Authorizing `identity:delete_project(project_id=20dbba5a31cd4d77a878e766a5aab0d0)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:28:59.358063 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed987a9-6392-4c03-94cb-83ac258d5397 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:28:59.389035 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f106c6-003f-440c-ab09-6510a44c377a admin admin] RBAC: Authorizing `identity:create_grant(project_id=9dda1c7250c7408593271d4252698e79, user_id=57d46739f2e548d9b5e7de3f54b64aea, role_id=136cdf78e00a426997dd4e37faff489b)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:28:59.391792 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f106c6-003f-440c-ab09-6510a44c377a admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:28:59.479816 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 957/1908] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 24 09:28:59 2022] PUT /identity/v3/projects/9dda1c7250c7408593271d4252698e79/users/57d46739f2e548d9b5e7de3f54b64aea/roles/136cdf78e00a426997dd4e37faff489b => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:28:59.491078 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-aed987a9-6392-4c03-94cb-83ac258d5397 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '20dbba5a31cd4d77a878e766a5aab0d0', 'request_id': 'req-aed987a9-6392-4c03-94cb-83ac258d5397'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:28:59.491745 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b51dbc3-1602-4041-a409-7efbaec62a2a None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:59.492117 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b51dbc3-1602-4041-a409-7efbaec62a2a None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:59.492651 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b51dbc3-1602-4041-a409-7efbaec62a2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:59.501460 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-aed987a9-6392-4c03-94cb-83ac258d5397 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '20dbba5a31cd4d77a878e766a5aab0d0', 'request_id': 'req-aed987a9-6392-4c03-94cb-83ac258d5397'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:28:59.507908 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-aed987a9-6392-4c03-94cb-83ac258d5397 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '20dbba5a31cd4d77a878e766a5aab0d0', 'request_id': 'req-aed987a9-6392-4c03-94cb-83ac258d5397'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:28:59.522047 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-8b51dbc3-1602-4041-a409-7efbaec62a2a None None] MFA Rules not processed for user `57d46739f2e548d9b5e7de3f54b64aea`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:28:59.523232 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-8b51dbc3-1602-4041-a409-7efbaec62a2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:28:59.530290 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:28:59.581126 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-aed987a9-6392-4c03-94cb-83ac258d5397 admin admin] The token cache is being invalidate because project 20dbba5a31cd4d77a878e766a5aab0d0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:28:59.581416 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-aed987a9-6392-4c03-94cb-83ac258d5397 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:28:59.582053 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-aed987a9-6392-4c03-94cb-83ac258d5397 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:28:59.583569 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 952/1909] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 24 09:28:59 2022] DELETE /identity/v3/projects/20dbba5a31cd4d77a878e766a5aab0d0 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:28:59.588371 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c138f511-57e7-4440-bdda-893497d41c32 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:59.588653 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c138f511-57e7-4440-bdda-893497d41c32 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:59.588926 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c138f511-57e7-4440-bdda-893497d41c32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:59.589224 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 958/1910] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 09:28:59 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 09:28:59.593808 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b51dbc3-1602-4041-a409-7efbaec62a2a None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:28:59.609446 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-c138f511-57e7-4440-bdda-893497d41c32 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:28:59.610676 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-c138f511-57e7-4440-bdda-893497d41c32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:28:59.671971 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b51dbc3-1602-4041-a409-7efbaec62a2a None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:28:59.676924 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf741cb-b7f7-4d4b-a387-cdca07643643 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:28:59.677535 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf741cb-b7f7-4d4b-a387-cdca07643643 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:59.677877 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf741cb-b7f7-4d4b-a387-cdca07643643 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:59.678079 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf741cb-b7f7-4d4b-a387-cdca07643643 admin admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:59.678782 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf741cb-b7f7-4d4b-a387-cdca07643643 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:28:59.680689 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf741cb-b7f7-4d4b-a387-cdca07643643 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:28:59.695814 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 953/1911] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 09:28:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 09:28:59.695814 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c138f511-57e7-4440-bdda-893497d41c32 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:28:59.696427 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-c138f511-57e7-4440-bdda-893497d41c32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:28:59.709134 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c138f511-57e7-4440-bdda-893497d41c32 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:28:59.715195 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737cecb1-08d8-4a24-b56d-a9c16c8af2bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:28:59.716398 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737cecb1-08d8-4a24-b56d-a9c16c8af2bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:59.722876 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737cecb1-08d8-4a24-b56d-a9c16c8af2bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:59.722876 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737cecb1-08d8-4a24-b56d-a9c16c8af2bb admin admin] PATH_INFO: `/v3/projects/972df1958193447e835cb13662304cc1` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:59.722876 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737cecb1-08d8-4a24-b56d-a9c16c8af2bb admin admin] RBAC: Authorizing `identity:delete_project(project_id=972df1958193447e835cb13662304cc1)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:28:59.722876 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737cecb1-08d8-4a24-b56d-a9c16c8af2bb admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:28:59.785378 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 959/1912] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 24 09:28:59 2022] POST /identity/v3/projects => generated 365 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 09:28:59.789819 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10d61c8-9862-4858-baf8-f3d6694af8a3 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:59.795818 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10d61c8-9862-4858-baf8-f3d6694af8a3 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:59.795818 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10d61c8-9862-4858-baf8-f3d6694af8a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:59.815279 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-f10d61c8-9862-4858-baf8-f3d6694af8a3 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:28:59.816579 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-f10d61c8-9862-4858-baf8-f3d6694af8a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:28:59.825539 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-737cecb1-08d8-4a24-b56d-a9c16c8af2bb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '972df1958193447e835cb13662304cc1', 'request_id': 'req-737cecb1-08d8-4a24-b56d-a9c16c8af2bb'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:28:59.835182 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-737cecb1-08d8-4a24-b56d-a9c16c8af2bb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '972df1958193447e835cb13662304cc1', 'request_id': 'req-737cecb1-08d8-4a24-b56d-a9c16c8af2bb'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:28:59.845762 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-737cecb1-08d8-4a24-b56d-a9c16c8af2bb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '972df1958193447e835cb13662304cc1', 'request_id': 'req-737cecb1-08d8-4a24-b56d-a9c16c8af2bb'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:28:59.879360 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 960/1913] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 09:28:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:28:59.882690 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10d61c8-9862-4858-baf8-f3d6694af8a3 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:28:59.882690 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-f10d61c8-9862-4858-baf8-f3d6694af8a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:28:59.894133 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10d61c8-9862-4858-baf8-f3d6694af8a3 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:28:59.898906 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65698b54-833d-46da-bb1e-6beef3b6f20b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:28:59.899523 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65698b54-833d-46da-bb1e-6beef3b6f20b admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:59.899835 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65698b54-833d-46da-bb1e-6beef3b6f20b admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:59.900111 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65698b54-833d-46da-bb1e-6beef3b6f20b admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:59.900739 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65698b54-833d-46da-bb1e-6beef3b6f20b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:28:59.902634 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65698b54-833d-46da-bb1e-6beef3b6f20b admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:28:59.911546 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-737cecb1-08d8-4a24-b56d-a9c16c8af2bb admin admin] The token cache is being invalidate because project 972df1958193447e835cb13662304cc1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:28:59.911955 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-737cecb1-08d8-4a24-b56d-a9c16c8af2bb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:28:59.913048 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-737cecb1-08d8-4a24-b56d-a9c16c8af2bb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:28:59.928088 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 954/1914] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 24 09:28:59 2022] DELETE /identity/v3/projects/972df1958193447e835cb13662304cc1 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:28:59.929008 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737cecb1-08d8-4a24-b56d-a9c16c8af2bb admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:28:59.930817 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-737cecb1-08d8-4a24-b56d-a9c16c8af2bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:28:59.943992 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737cecb1-08d8-4a24-b56d-a9c16c8af2bb admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:28:59.948749 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc87089-3437-49fd-a7df-349629107182 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:28:59.949343 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc87089-3437-49fd-a7df-349629107182 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:59.949625 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc87089-3437-49fd-a7df-349629107182 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:59.950449 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc87089-3437-49fd-a7df-349629107182 admin admin] PATH_INFO: `/v3/domains` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:59.951091 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc87089-3437-49fd-a7df-349629107182 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:28:59.951397 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc87089-3437-49fd-a7df-349629107182 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:28:59.953169 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc87089-3437-49fd-a7df-349629107182 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:28:59.965236 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 955/1915] 10.222.0.16 () {58 vars in 1153 bytes} [Wed Aug 24 09:28:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:28:59.967291 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f82e365-1a21-4d90-95d4-ffd2ea1fd62b None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:28:59.967611 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f82e365-1a21-4d90-95d4-ffd2ea1fd62b None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:28:59.968305 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f82e365-1a21-4d90-95d4-ffd2ea1fd62b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:28:59.986179 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 961/1916] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 24 09:28:59 2022] POST /identity/v3/users => generated 289 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:28:59.986179 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65698b54-833d-46da-bb1e-6beef3b6f20b admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:28:59.990196 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-8f82e365-1a21-4d90-95d4-ffd2ea1fd62b None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:28:59.991427 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-8f82e365-1a21-4d90-95d4-ffd2ea1fd62b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:28:59.994160 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65698b54-833d-46da-bb1e-6beef3b6f20b admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:28:59.998891 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c46f3c4-3859-4098-a2e3-189ca2c5de1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:29:00.007093 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 956/1917] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 09:28:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:29:00.007253 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c46f3c4-3859-4098-a2e3-189ca2c5de1a admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:29:00.007253 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c46f3c4-3859-4098-a2e3-189ca2c5de1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:29:00.007253 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c46f3c4-3859-4098-a2e3-189ca2c5de1a admin admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:29:00.010602 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f82e365-1a21-4d90-95d4-ffd2ea1fd62b None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:29:00.010706 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c46f3c4-3859-4098-a2e3-189ca2c5de1a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:29:00.010803 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-8f82e365-1a21-4d90-95d4-ffd2ea1fd62b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:29:00.010891 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c46f3c4-3859-4098-a2e3-189ca2c5de1a admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:29:00.013507 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f82e365-1a21-4d90-95d4-ffd2ea1fd62b None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:29:00.018164 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb728ac-e14b-49c5-9406-482af0ff17f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:29:00.018770 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb728ac-e14b-49c5-9406-482af0ff17f2 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:29:00.019045 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb728ac-e14b-49c5-9406-482af0ff17f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:29:00.019319 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb728ac-e14b-49c5-9406-482af0ff17f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:29:00.020008 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb728ac-e14b-49c5-9406-482af0ff17f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:29:00.021625 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb728ac-e14b-49c5-9406-482af0ff17f2 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:29:00.031812 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 957/1918] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 24 09:28:59 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:29:00.033061 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9bd916-5ba6-49fb-b36a-0d0c94641e29 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:29:00.033492 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9bd916-5ba6-49fb-b36a-0d0c94641e29 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:29:00.033586 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9bd916-5ba6-49fb-b36a-0d0c94641e29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:29:00.046123 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:29:00.053637 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-8f9bd916-5ba6-49fb-b36a-0d0c94641e29 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:29:00.054843 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-8f9bd916-5ba6-49fb-b36a-0d0c94641e29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:29:00.063541 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 958/1919] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 09:29:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:29:00.064350 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9bd916-5ba6-49fb-b36a-0d0c94641e29 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:29:00.073615 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9bd916-5ba6-49fb-b36a-0d0c94641e29 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:29:00.085658 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 962/1920] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 24 09:28:59 2022] POST /identity/v3/projects => generated 373 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:29:00.086500 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c46f3c4-3859-4098-a2e3-189ca2c5de1a admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:29:00.088469 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f35a8aa-a6bc-4567-85ef-553138e89f6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:29:00.089148 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f35a8aa-a6bc-4567-85ef-553138e89f6b admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:29:00.089425 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f35a8aa-a6bc-4567-85ef-553138e89f6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:29:00.091137 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f35a8aa-a6bc-4567-85ef-553138e89f6b admin admin] PATH_INFO: `/v3/projects/4a2351b8835d4d4caea6dcc8e50e188a/users/cb9c4b6cc91e411a84416f4f491363c1/roles/15f634d4a2d247ddab90c458e5e1c141` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:29:00.111679 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f35a8aa-a6bc-4567-85ef-553138e89f6b admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a2351b8835d4d4caea6dcc8e50e188a, user_id=cb9c4b6cc91e411a84416f4f491363c1, role_id=15f634d4a2d247ddab90c458e5e1c141)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:29:00.114719 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f35a8aa-a6bc-4567-85ef-553138e89f6b admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:29:00.178216 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c46f3c4-3859-4098-a2e3-189ca2c5de1a admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:29:00.183316 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 959/1921] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 24 09:29:00 2022] PUT /identity/v3/projects/4a2351b8835d4d4caea6dcc8e50e188a/users/cb9c4b6cc91e411a84416f4f491363c1/roles/15f634d4a2d247ddab90c458e5e1c141 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:29:00.189431 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f35a8aa-a6bc-4567-85ef-553138e89f6b admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:29:00.268516 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f35a8aa-a6bc-4567-85ef-553138e89f6b admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:29:00.274071 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef44a1c-02e3-4b3d-9d94-00d7ad6d151c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:29:00.274833 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef44a1c-02e3-4b3d-9d94-00d7ad6d151c admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:29:00.275172 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef44a1c-02e3-4b3d-9d94-00d7ad6d151c admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:29:00.275452 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef44a1c-02e3-4b3d-9d94-00d7ad6d151c admin admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:29:00.276136 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef44a1c-02e3-4b3d-9d94-00d7ad6d151c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:29:00.278807 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef44a1c-02e3-4b3d-9d94-00d7ad6d151c admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:29:00.296912 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dcfbc8-58cb-4903-89e4-f75a4eff45cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:29:00.302969 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dcfbc8-58cb-4903-89e4-f75a4eff45cc admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:29:00.303262 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dcfbc8-58cb-4903-89e4-f75a4eff45cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:29:00.303544 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dcfbc8-58cb-4903-89e4-f75a4eff45cc admin admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:29:00.304248 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65dcfbc8-58cb-4903-89e4-f75a4eff45cc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:29:00.304574 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65dcfbc8-58cb-4903-89e4-f75a4eff45cc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:29:00.306462 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65dcfbc8-58cb-4903-89e4-f75a4eff45cc admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:29:00.319420 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 963/1922] 10.222.0.16 () {58 vars in 1153 bytes} [Wed Aug 24 09:29:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:29:00.320313 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dcfbc8-58cb-4903-89e4-f75a4eff45cc admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:29:00.339149 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dcfbc8-58cb-4903-89e4-f75a4eff45cc admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:29:00.344935 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee898d3-7eae-4267-b3bd-02f7e72cbcf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:29:00.346170 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee898d3-7eae-4267-b3bd-02f7e72cbcf3 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:29:00.346170 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee898d3-7eae-4267-b3bd-02f7e72cbcf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:29:00.346349 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee898d3-7eae-4267-b3bd-02f7e72cbcf3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:29:00.346911 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee898d3-7eae-4267-b3bd-02f7e72cbcf3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:29:00.348553 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee898d3-7eae-4267-b3bd-02f7e72cbcf3 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:29:00.361498 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 964/1923] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 24 09:29:00 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:29:00.363209 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee898d3-7eae-4267-b3bd-02f7e72cbcf3 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:29:00.374570 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee898d3-7eae-4267-b3bd-02f7e72cbcf3 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:29:00.379407 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58cb0ec-7cf1-485c-833b-37914476ce82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:29:00.380028 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58cb0ec-7cf1-485c-833b-37914476ce82 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:29:00.380306 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58cb0ec-7cf1-485c-833b-37914476ce82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:29:00.380585 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58cb0ec-7cf1-485c-833b-37914476ce82 admin admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:29:00.390848 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58cb0ec-7cf1-485c-833b-37914476ce82 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:29:00.392745 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58cb0ec-7cf1-485c-833b-37914476ce82 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:29:00.398859 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 960/1924] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 24 09:29:00 2022] POST /identity/v3/users => generated 287 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:29:00.399827 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef44a1c-02e3-4b3d-9d94-00d7ad6d151c admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:29:00.409544 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef44a1c-02e3-4b3d-9d94-00d7ad6d151c admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:29:00.414933 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faff618f-0d40-484a-9c17-435cb6a85707 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:29:00.415609 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faff618f-0d40-484a-9c17-435cb6a85707 admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:29:00.415911 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faff618f-0d40-484a-9c17-435cb6a85707 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:29:00.416188 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faff618f-0d40-484a-9c17-435cb6a85707 admin admin] PATH_INFO: `/v3/projects/4a2351b8835d4d4caea6dcc8e50e188a/users/cb9c4b6cc91e411a84416f4f491363c1/roles/15f634d4a2d247ddab90c458e5e1c141` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:29:00.418278 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faff618f-0d40-484a-9c17-435cb6a85707 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a2351b8835d4d4caea6dcc8e50e188a, user_id=cb9c4b6cc91e411a84416f4f491363c1, role_id=15f634d4a2d247ddab90c458e5e1c141)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:29:00.421012 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faff618f-0d40-484a-9c17-435cb6a85707 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:29:00.433466 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 961/1925] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 24 09:29:00 2022] PUT /identity/v3/projects/4a2351b8835d4d4caea6dcc8e50e188a/users/cb9c4b6cc91e411a84416f4f491363c1/roles/15f634d4a2d247ddab90c458e5e1c141 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:29:00.437918 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faff618f-0d40-484a-9c17-435cb6a85707 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:29:00.531193 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faff618f-0d40-484a-9c17-435cb6a85707 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:29:00.553181 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6edf56b-7fc7-419d-a2f0-00fdf7e7370c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:29:00.553181 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6edf56b-7fc7-419d-a2f0-00fdf7e7370c admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:29:00.553181 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6edf56b-7fc7-419d-a2f0-00fdf7e7370c admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:29:00.553181 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6edf56b-7fc7-419d-a2f0-00fdf7e7370c admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:29:00.553860 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6edf56b-7fc7-419d-a2f0-00fdf7e7370c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:29:00.553977 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 965/1926] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 24 09:29:00 2022] POST /identity/v3/projects => generated 377 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:29:00.554760 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6edf56b-7fc7-419d-a2f0-00fdf7e7370c admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:29:00.554890 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a1e03d-dd2f-4cd7-bb04-5a4687f2ef78 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:29:00.554890 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a1e03d-dd2f-4cd7-bb04-5a4687f2ef78 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:29:00.554890 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a1e03d-dd2f-4cd7-bb04-5a4687f2ef78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:29:00.555790 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 962/1927] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 24 09:29:00 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:29:00.555790 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6edf56b-7fc7-419d-a2f0-00fdf7e7370c admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:29:00.566118 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6edf56b-7fc7-419d-a2f0-00fdf7e7370c admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:29:00.577500 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-b5a1e03d-dd2f-4cd7-bb04-5a4687f2ef78 None None] MFA Rules not processed for user `cb9c4b6cc91e411a84416f4f491363c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:29:00.578733 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-b5a1e03d-dd2f-4cd7-bb04-5a4687f2ef78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:29:00.625136 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf9d187-fcd8-420f-ab40-ec22c576b01a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:29:00.625749 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf9d187-fcd8-420f-ab40-ec22c576b01a admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:29:00.626032 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf9d187-fcd8-420f-ab40-ec22c576b01a admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:29:00.626309 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf9d187-fcd8-420f-ab40-ec22c576b01a admin admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:29:00.627001 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf9d187-fcd8-420f-ab40-ec22c576b01a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:29:00.628989 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf9d187-fcd8-420f-ab40-ec22c576b01a admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:29:00.644078 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:29:00.651333 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 966/1928] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 09:29:00 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:29:00.652478 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a1e03d-dd2f-4cd7-bb04-5a4687f2ef78 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:29:00.670380 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a1e03d-dd2f-4cd7-bb04-5a4687f2ef78 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:29:00.675335 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3e5367-df6a-42b7-b955-02cc3ce8f752 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:29:00.676007 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3e5367-df6a-42b7-b955-02cc3ce8f752 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:29:00.676292 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3e5367-df6a-42b7-b955-02cc3ce8f752 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:29:00.676633 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3e5367-df6a-42b7-b955-02cc3ce8f752 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:29:00.677807 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3e5367-df6a-42b7-b955-02cc3ce8f752 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:29:00.679692 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3e5367-df6a-42b7-b955-02cc3ce8f752 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:29:00.690475 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 967/1929] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 24 09:29:00 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:29:00.691042 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3e5367-df6a-42b7-b955-02cc3ce8f752 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:29:00.692767 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-5b3e5367-df6a-42b7-b955-02cc3ce8f752 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:29:00.702738 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 963/1930] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 24 09:29:00 2022] POST /identity/v3/users => generated 289 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:29:00.706468 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf9d187-fcd8-420f-ab40-ec22c576b01a admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:29:00.722124 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf9d187-fcd8-420f-ab40-ec22c576b01a admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:29:00.742783 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3ac61d-87db-416f-9bed-ffb8e85b815f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:29:00.742783 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3ac61d-87db-416f-9bed-ffb8e85b815f admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:29:00.742783 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3ac61d-87db-416f-9bed-ffb8e85b815f admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:29:00.742783 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3ac61d-87db-416f-9bed-ffb8e85b815f admin admin] PATH_INFO: `/v3/projects/786299295cf944b2b4fc909b9dbe72a4/users/35e4f58d358e48fe83677a3df6a083b1/roles/136cdf78e00a426997dd4e37faff489b` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:29:00.758863 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3ac61d-87db-416f-9bed-ffb8e85b815f admin admin] RBAC: Authorizing `identity:create_grant(project_id=786299295cf944b2b4fc909b9dbe72a4, user_id=35e4f58d358e48fe83677a3df6a083b1, role_id=136cdf78e00a426997dd4e37faff489b)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:29:00.761739 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3ac61d-87db-416f-9bed-ffb8e85b815f admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:29:00.770993 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3e5367-df6a-42b7-b955-02cc3ce8f752 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:29:00.774927 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4193ced6-ee30-4514-a22b-db23f576aaba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c32b83577c46488906fe409a790b24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:29:00.777544 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4193ced6-ee30-4514-a22b-db23f576aaba service cinder] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:29:00.777544 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4193ced6-ee30-4514-a22b-db23f576aaba service cinder] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:29:00.777544 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4193ced6-ee30-4514-a22b-db23f576aaba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:29:00.787212 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4193ced6-ee30-4514-a22b-db23f576aaba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:29:00.788825 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4193ced6-ee30-4514-a22b-db23f576aaba service cinder] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:29:00.798990 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 968/1931] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:29:00 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:29:00.800536 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4193ced6-ee30-4514-a22b-db23f576aaba service cinder] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:29:00.811059 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4193ced6-ee30-4514-a22b-db23f576aaba service cinder] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:29:00.816926 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6ec2d9-9f94-44f6-830c-7ac2aa0f6a18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:29:00.817838 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6ec2d9-9f94-44f6-830c-7ac2aa0f6a18 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:29:00.818125 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6ec2d9-9f94-44f6-830c-7ac2aa0f6a18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:29:00.818404 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6ec2d9-9f94-44f6-830c-7ac2aa0f6a18 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:29:00.819079 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6ec2d9-9f94-44f6-830c-7ac2aa0f6a18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:29:00.820724 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6ec2d9-9f94-44f6-830c-7ac2aa0f6a18 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:29:00.832256 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 969/1932] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 24 09:29:00 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:29:00.843316 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 964/1933] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 24 09:29:00 2022] PUT /identity/v3/projects/786299295cf944b2b4fc909b9dbe72a4/users/35e4f58d358e48fe83677a3df6a083b1/roles/136cdf78e00a426997dd4e37faff489b => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:29:00.843460 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6ec2d9-9f94-44f6-830c-7ac2aa0f6a18 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:29:00.845521 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2b1de5-3705-46c3-9577-4980a14b3fe7 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:29:00.846053 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2b1de5-3705-46c3-9577-4980a14b3fe7 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:29:00.846334 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2b1de5-3705-46c3-9577-4980a14b3fe7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:29:00.847151 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6ec2d9-9f94-44f6-830c-7ac2aa0f6a18 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:29:00.884313 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-3e2b1de5-3705-46c3-9577-4980a14b3fe7 None None] MFA Rules not processed for user `35e4f58d358e48fe83677a3df6a083b1`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:29:00.885667 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-3e2b1de5-3705-46c3-9577-4980a14b3fe7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:29:00.919802 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cad0d9-b28f-422e-be9e-78042b31c87e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:29:00.920438 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cad0d9-b28f-422e-be9e-78042b31c87e admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:29:00.920726 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cad0d9-b28f-422e-be9e-78042b31c87e admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:29:00.921004 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cad0d9-b28f-422e-be9e-78042b31c87e admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:29:00.922020 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64cad0d9-b28f-422e-be9e-78042b31c87e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:29:00.926426 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64cad0d9-b28f-422e-be9e-78042b31c87e admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:29:00.960449 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 970/1934] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 24 09:29:00 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 09:29:00.963295 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cad0d9-b28f-422e-be9e-78042b31c87e admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:29:00.977793 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cad0d9-b28f-422e-be9e-78042b31c87e admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:29:00.987372 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65724e41-2950-400a-9450-caf920d4c223 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:29:00.988229 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65724e41-2950-400a-9450-caf920d4c223 admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:29:00.988525 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65724e41-2950-400a-9450-caf920d4c223 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:29:00.988858 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65724e41-2950-400a-9450-caf920d4c223 admin admin] PATH_INFO: `/v3/projects/83b7398cf2994a328ae9ba896956a6fe/users/0525c1879f3a4b44a131daac894680a0/roles/136cdf78e00a426997dd4e37faff489b` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:29:01.015353 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65724e41-2950-400a-9450-caf920d4c223 admin admin] RBAC: Authorizing `identity:create_grant(project_id=83b7398cf2994a328ae9ba896956a6fe, user_id=0525c1879f3a4b44a131daac894680a0, role_id=136cdf78e00a426997dd4e37faff489b)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:29:01.022391 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65724e41-2950-400a-9450-caf920d4c223 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:29:01.027153 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 965/1935] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 09:29:00 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 09:29:01.044420 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2b1de5-3705-46c3-9577-4980a14b3fe7 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:29:01.065711 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2b1de5-3705-46c3-9577-4980a14b3fe7 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:29:01.074321 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6771b10-738a-40d8-81a1-21319ec4d4f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:29:01.079733 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6771b10-738a-40d8-81a1-21319ec4d4f3 admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:29:01.079733 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6771b10-738a-40d8-81a1-21319ec4d4f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:29:01.079733 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6771b10-738a-40d8-81a1-21319ec4d4f3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:29:01.079733 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6771b10-738a-40d8-81a1-21319ec4d4f3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:29:01.080698 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6771b10-738a-40d8-81a1-21319ec4d4f3 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:29:01.101594 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 971/1936] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 24 09:29:00 2022] PUT /identity/v3/projects/83b7398cf2994a328ae9ba896956a6fe/users/0525c1879f3a4b44a131daac894680a0/roles/136cdf78e00a426997dd4e37faff489b => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:29:01.107393 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65724e41-2950-400a-9450-caf920d4c223 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:29:01.107393 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-65724e41-2950-400a-9450-caf920d4c223 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:29:01.179671 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65724e41-2950-400a-9450-caf920d4c223 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:29:01.183429 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769dd186-6582-4d9e-a262-143f89466f4a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c32b83577c46488906fe409a790b24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:29:01.184227 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769dd186-6582-4d9e-a262-143f89466f4a service cinder] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:29:01.184486 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769dd186-6582-4d9e-a262-143f89466f4a service cinder] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:29:01.184906 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769dd186-6582-4d9e-a262-143f89466f4a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:29:01.186728 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-769dd186-6582-4d9e-a262-143f89466f4a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:29:01.219718 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 966/1937] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 24 09:29:01 2022] POST /identity/v3/projects => generated 375 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:29:01.219718 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8c5ba6-1215-4805-b6b8-5f5d788f07c1 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:29:01.219718 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8c5ba6-1215-4805-b6b8-5f5d788f07c1 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:29:01.219718 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8c5ba6-1215-4805-b6b8-5f5d788f07c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:29:01.248763 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769dd186-6582-4d9e-a262-143f89466f4a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:29:01.249150 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-1b8c5ba6-1215-4805-b6b8-5f5d788f07c1 None None] MFA Rules not processed for user `0525c1879f3a4b44a131daac894680a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:29:01.250205 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769dd186-6582-4d9e-a262-143f89466f4a service cinder] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:29:01.250552 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-1b8c5ba6-1215-4805-b6b8-5f5d788f07c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:29:01.259902 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 972/1938] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:29:01 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:29:01.260876 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769dd186-6582-4d9e-a262-143f89466f4a service cinder] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:29:01.338383 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 967/1939] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 09:29:01 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:29:01.350882 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8c5ba6-1215-4805-b6b8-5f5d788f07c1 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:29:01.354876 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769dd186-6582-4d9e-a262-143f89466f4a service cinder] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:29:01.407660 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:29:01.418373 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8c5ba6-1215-4805-b6b8-5f5d788f07c1 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:29:01.424268 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b2be74-405d-4bc7-bac6-c311468e03e7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c32b83577c46488906fe409a790b24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:29:01.425188 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616725fe-50c7-4d5c-973b-9521a0157c4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:29:01.425821 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616725fe-50c7-4d5c-973b-9521a0157c4b admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:29:01.425938 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b2be74-405d-4bc7-bac6-c311468e03e7 service cinder] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:29:01.426243 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616725fe-50c7-4d5c-973b-9521a0157c4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:29:01.426353 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616725fe-50c7-4d5c-973b-9521a0157c4b admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:29:01.427011 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616725fe-50c7-4d5c-973b-9521a0157c4b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:29:01.435856 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616725fe-50c7-4d5c-973b-9521a0157c4b admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:29:01.436130 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b2be74-405d-4bc7-bac6-c311468e03e7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:29:01.436130 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b2be74-405d-4bc7-bac6-c311468e03e7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:29:01.447920 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b2be74-405d-4bc7-bac6-c311468e03e7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:29:01.449386 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b2be74-405d-4bc7-bac6-c311468e03e7 service cinder] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:29:01.460569 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 968/1940] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:29:01 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 09:29:01.548513 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 973/1941] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 24 09:29:01 2022] POST /identity/v3/users => generated 294 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:29:01.553614 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b2be74-405d-4bc7-bac6-c311468e03e7 service cinder] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:29:01.566265 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b2be74-405d-4bc7-bac6-c311468e03e7 service cinder] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:29:01.573780 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329ec56b-d545-4c59-ab72-35d1ef8d5b41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:29:01.573780 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329ec56b-d545-4c59-ab72-35d1ef8d5b41 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:29:01.573780 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329ec56b-d545-4c59-ab72-35d1ef8d5b41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:29:01.573780 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329ec56b-d545-4c59-ab72-35d1ef8d5b41 admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:29:01.574100 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329ec56b-d545-4c59-ab72-35d1ef8d5b41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:29:01.575973 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329ec56b-d545-4c59-ab72-35d1ef8d5b41 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:29:01.592915 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 969/1942] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 24 09:29:01 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:29:01.593355 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616725fe-50c7-4d5c-973b-9521a0157c4b admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:29:01.615961 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616725fe-50c7-4d5c-973b-9521a0157c4b admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:29:01.621202 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c706e8-6cea-4275-aa43-4e8a2b38c7db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:29:01.622467 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c706e8-6cea-4275-aa43-4e8a2b38c7db admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:29:01.622864 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c706e8-6cea-4275-aa43-4e8a2b38c7db admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:29:01.623301 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c706e8-6cea-4275-aa43-4e8a2b38c7db admin admin] PATH_INFO: `/v3/projects/40f66d2f21354a7b87c731fe40cfee23/users/59833b44efa644f1b030101f63c6d939/roles/15f634d4a2d247ddab90c458e5e1c141` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:29:01.640668 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c706e8-6cea-4275-aa43-4e8a2b38c7db admin admin] RBAC: Authorizing `identity:create_grant(project_id=40f66d2f21354a7b87c731fe40cfee23, user_id=59833b44efa644f1b030101f63c6d939, role_id=15f634d4a2d247ddab90c458e5e1c141)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:29:01.643337 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c706e8-6cea-4275-aa43-4e8a2b38c7db admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:29:01.759781 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 974/1943] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 24 09:29:01 2022] PUT /identity/v3/projects/40f66d2f21354a7b87c731fe40cfee23/users/59833b44efa644f1b030101f63c6d939/roles/15f634d4a2d247ddab90c458e5e1c141 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:29:01.764169 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329ec56b-d545-4c59-ab72-35d1ef8d5b41 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:29:01.775231 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329ec56b-d545-4c59-ab72-35d1ef8d5b41 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:29:01.808921 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:29:01.838521 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37645e1-e3ab-4a5b-9416-d5ff27f09b96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:29:01.839167 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37645e1-e3ab-4a5b-9416-d5ff27f09b96 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:29:01.839446 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37645e1-e3ab-4a5b-9416-d5ff27f09b96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:29:01.839763 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37645e1-e3ab-4a5b-9416-d5ff27f09b96 admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:29:01.840452 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37645e1-e3ab-4a5b-9416-d5ff27f09b96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:29:01.842035 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37645e1-e3ab-4a5b-9416-d5ff27f09b96 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:29:01.852010 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 970/1944] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 24 09:29:01 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:29:01.856478 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c706e8-6cea-4275-aa43-4e8a2b38c7db admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:29:01.866892 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c706e8-6cea-4275-aa43-4e8a2b38c7db admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:29:01.872158 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333c74e5-bfa4-4b48-8354-1691ada7755d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:29:01.872807 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333c74e5-bfa4-4b48-8354-1691ada7755d admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:29:01.873086 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333c74e5-bfa4-4b48-8354-1691ada7755d admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:29:01.873357 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333c74e5-bfa4-4b48-8354-1691ada7755d admin admin] PATH_INFO: `/v3/projects/40f66d2f21354a7b87c731fe40cfee23/users/59833b44efa644f1b030101f63c6d939/roles/15f634d4a2d247ddab90c458e5e1c141` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:29:01.875621 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333c74e5-bfa4-4b48-8354-1691ada7755d admin admin] RBAC: Authorizing `identity:create_grant(project_id=40f66d2f21354a7b87c731fe40cfee23, user_id=59833b44efa644f1b030101f63c6d939, role_id=15f634d4a2d247ddab90c458e5e1c141)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:29:01.878237 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333c74e5-bfa4-4b48-8354-1691ada7755d admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:29:01.889858 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 975/1945] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 24 09:29:01 2022] PUT /identity/v3/projects/40f66d2f21354a7b87c731fe40cfee23/users/59833b44efa644f1b030101f63c6d939/roles/15f634d4a2d247ddab90c458e5e1c141 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:29:01.900887 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2dcb8d-15df-470a-a7c3-7960bf00f81c None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:29:01.901182 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2dcb8d-15df-470a-a7c3-7960bf00f81c None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:29:01.901488 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2dcb8d-15df-470a-a7c3-7960bf00f81c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:29:01.936057 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-8a2dcb8d-15df-470a-a7c3-7960bf00f81c None None] MFA Rules not processed for user `59833b44efa644f1b030101f63c6d939`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:29:01.937377 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-8a2dcb8d-15df-470a-a7c3-7960bf00f81c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:29:02.015647 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 971/1946] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 09:29:01 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:29:02.028564 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333c74e5-bfa4-4b48-8354-1691ada7755d admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:29:02.086497 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333c74e5-bfa4-4b48-8354-1691ada7755d admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:29:02.090472 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269417bb-9370-4a50-a2a4-6caa8d0504c8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c32b83577c46488906fe409a790b24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:29:02.091071 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269417bb-9370-4a50-a2a4-6caa8d0504c8 service cinder] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:29:02.091347 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269417bb-9370-4a50-a2a4-6caa8d0504c8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:29:02.091647 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269417bb-9370-4a50-a2a4-6caa8d0504c8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:29:02.176203 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269417bb-9370-4a50-a2a4-6caa8d0504c8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:29:02.178357 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269417bb-9370-4a50-a2a4-6caa8d0504c8 service cinder] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:29:02.192999 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 976/1947] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:29:02 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:29:02.221819 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:INSERT=6,keystone:DELETE=1,keystone:SELECT=93 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:29:02.719940 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:29:03.049938 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:29:03.386825 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:29:03.713484 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:29:03.929975 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:29:04.432626 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:29:04.778520 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:29:05.318124 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2dcb8d-15df-470a-a7c3-7960bf00f81c None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:29:05.392234 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2dcb8d-15df-470a-a7c3-7960bf00f81c None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:29:05.397922 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2743fddb-f291-46cf-bd70-d3653ab7048a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:29:05.398961 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2743fddb-f291-46cf-bd70-d3653ab7048a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:29:05.398961 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2743fddb-f291-46cf-bd70-d3653ab7048a admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:29:05.399177 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2743fddb-f291-46cf-bd70-d3653ab7048a admin admin] PATH_INFO: `/v3/users/57d46739f2e548d9b5e7de3f54b64aea` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:29:05.400477 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2743fddb-f291-46cf-bd70-d3653ab7048a admin admin] RBAC: Authorizing `identity:delete_user(user_id=57d46739f2e548d9b5e7de3f54b64aea)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:29:05.402522 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2743fddb-f291-46cf-bd70-d3653ab7048a admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:29:05.613916 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-2743fddb-f291-46cf-bd70-d3653ab7048a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '57d46739f2e548d9b5e7de3f54b64aea'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:29:05.614578 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-2743fddb-f291-46cf-bd70-d3653ab7048a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '57d46739f2e548d9b5e7de3f54b64aea'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:29:05.629754 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-2743fddb-f291-46cf-bd70-d3653ab7048a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '57d46739f2e548d9b5e7de3f54b64aea'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:29:05.636904 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-2743fddb-f291-46cf-bd70-d3653ab7048a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '57d46739f2e548d9b5e7de3f54b64aea'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:29:05.695463 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-2743fddb-f291-46cf-bd70-d3653ab7048a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '57d46739f2e548d9b5e7de3f54b64aea'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:29:05.703617 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 972/1948] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 24 09:29:05 2022] DELETE /identity/v3/users/57d46739f2e548d9b5e7de3f54b64aea => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:29:05.708043 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4220c40-a50b-4f98-b5ab-403d2753ad28 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:29:05.708642 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4220c40-a50b-4f98-b5ab-403d2753ad28 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:29:05.708919 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4220c40-a50b-4f98-b5ab-403d2753ad28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:29:05.734920 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-e4220c40-a50b-4f98-b5ab-403d2753ad28 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:29:05.736468 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-e4220c40-a50b-4f98-b5ab-403d2753ad28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:29:05.826403 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 977/1949] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 09:29:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:29:05.838271 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2743fddb-f291-46cf-bd70-d3653ab7048a admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:29:05.839995 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-2743fddb-f291-46cf-bd70-d3653ab7048a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:29:05.937078 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2743fddb-f291-46cf-bd70-d3653ab7048a admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:29:05.941151 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6886801d-6afc-4bba-911a-84433e9b0593 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e5260e798404e8bb12b25ca8ea97052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:29:05.942074 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6886801d-6afc-4bba-911a-84433e9b0593 service neutron] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:29:05.942500 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6886801d-6afc-4bba-911a-84433e9b0593 service neutron] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:29:05.943299 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6886801d-6afc-4bba-911a-84433e9b0593 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:29:05.953791 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6886801d-6afc-4bba-911a-84433e9b0593 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:29:05.955217 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6886801d-6afc-4bba-911a-84433e9b0593 service neutron] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:29:05.966638 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 973/1950] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:29:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:29:06.382430 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4220c40-a50b-4f98-b5ab-403d2753ad28 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:29:06.384048 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-e4220c40-a50b-4f98-b5ab-403d2753ad28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:29:06.403744 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4220c40-a50b-4f98-b5ab-403d2753ad28 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:29:06.424781 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1953d6b-f6ae-4e9f-b859-812c026f654c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:29:06.427852 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1953d6b-f6ae-4e9f-b859-812c026f654c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:29:06.427852 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1953d6b-f6ae-4e9f-b859-812c026f654c admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:29:06.427852 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1953d6b-f6ae-4e9f-b859-812c026f654c admin admin] PATH_INFO: `/v3/projects/9dda1c7250c7408593271d4252698e79` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:29:06.427852 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1953d6b-f6ae-4e9f-b859-812c026f654c admin admin] RBAC: Authorizing `identity:delete_project(project_id=9dda1c7250c7408593271d4252698e79)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:29:06.429325 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1953d6b-f6ae-4e9f-b859-812c026f654c admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:29:06.532970 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-f1953d6b-f6ae-4e9f-b859-812c026f654c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9dda1c7250c7408593271d4252698e79', 'request_id': 'req-f1953d6b-f6ae-4e9f-b859-812c026f654c'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:29:06.539980 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-f1953d6b-f6ae-4e9f-b859-812c026f654c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9dda1c7250c7408593271d4252698e79', 'request_id': 'req-f1953d6b-f6ae-4e9f-b859-812c026f654c'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:29:06.553268 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-f1953d6b-f6ae-4e9f-b859-812c026f654c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9dda1c7250c7408593271d4252698e79', 'request_id': 'req-f1953d6b-f6ae-4e9f-b859-812c026f654c'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:29:06.626413 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-f1953d6b-f6ae-4e9f-b859-812c026f654c admin admin] The token cache is being invalidate because project 9dda1c7250c7408593271d4252698e79 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81602) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:29:06.626725 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-f1953d6b-f6ae-4e9f-b859-812c026f654c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:29:06.627396 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-f1953d6b-f6ae-4e9f-b859-812c026f654c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:29:06.628944 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 978/1951] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 24 09:29:06 2022] DELETE /identity/v3/projects/9dda1c7250c7408593271d4252698e79 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:29:06.632931 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6886801d-6afc-4bba-911a-84433e9b0593 service neutron] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:29:06.642890 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6886801d-6afc-4bba-911a-84433e9b0593 service neutron] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:29:06.650822 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:29:06.699019 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f3856f-04ab-4be2-9f4d-c29228add8b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:29:06.699738 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f3856f-04ab-4be2-9f4d-c29228add8b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:29:06.700295 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f3856f-04ab-4be2-9f4d-c29228add8b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:29:06.700826 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f3856f-04ab-4be2-9f4d-c29228add8b9 admin admin] PATH_INFO: `/v3/users/cb9c4b6cc91e411a84416f4f491363c1` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:29:06.702389 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f3856f-04ab-4be2-9f4d-c29228add8b9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cb9c4b6cc91e411a84416f4f491363c1)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:29:06.704378 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f3856f-04ab-4be2-9f4d-c29228add8b9 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:29:06.910237 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-c4f3856f-04ab-4be2-9f4d-c29228add8b9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cb9c4b6cc91e411a84416f4f491363c1'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:29:06.912366 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-c4f3856f-04ab-4be2-9f4d-c29228add8b9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cb9c4b6cc91e411a84416f4f491363c1'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:29:06.924714 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-c4f3856f-04ab-4be2-9f4d-c29228add8b9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cb9c4b6cc91e411a84416f4f491363c1'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:29:06.931739 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-c4f3856f-04ab-4be2-9f4d-c29228add8b9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cb9c4b6cc91e411a84416f4f491363c1'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:29:06.980759 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-c4f3856f-04ab-4be2-9f4d-c29228add8b9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cb9c4b6cc91e411a84416f4f491363c1'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:29:06.988086 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 974/1952] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 24 09:29:06 2022] DELETE /identity/v3/users/cb9c4b6cc91e411a84416f4f491363c1 => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:29:07.416596 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1953d6b-f6ae-4e9f-b859-812c026f654c admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:29:07.418096 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-f1953d6b-f6ae-4e9f-b859-812c026f654c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:29:07.436362 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1953d6b-f6ae-4e9f-b859-812c026f654c admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:29:07.456874 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:29:07.501637 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05feb4a-5efd-4055-abd6-0e376125ebc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:29:07.502271 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05feb4a-5efd-4055-abd6-0e376125ebc4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:29:07.502551 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05feb4a-5efd-4055-abd6-0e376125ebc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:29:07.502823 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05feb4a-5efd-4055-abd6-0e376125ebc4 admin admin] PATH_INFO: `/v3/projects/4a2351b8835d4d4caea6dcc8e50e188a` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:29:07.503997 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05feb4a-5efd-4055-abd6-0e376125ebc4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a2351b8835d4d4caea6dcc8e50e188a)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:29:07.505944 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05feb4a-5efd-4055-abd6-0e376125ebc4 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:29:07.586877 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-c05feb4a-5efd-4055-abd6-0e376125ebc4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a2351b8835d4d4caea6dcc8e50e188a', 'request_id': 'req-c05feb4a-5efd-4055-abd6-0e376125ebc4'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:29:07.593211 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-c05feb4a-5efd-4055-abd6-0e376125ebc4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a2351b8835d4d4caea6dcc8e50e188a', 'request_id': 'req-c05feb4a-5efd-4055-abd6-0e376125ebc4'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:29:07.600364 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-c05feb4a-5efd-4055-abd6-0e376125ebc4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a2351b8835d4d4caea6dcc8e50e188a', 'request_id': 'req-c05feb4a-5efd-4055-abd6-0e376125ebc4'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:29:07.688647 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-c05feb4a-5efd-4055-abd6-0e376125ebc4 admin admin] The token cache is being invalidate because project 4a2351b8835d4d4caea6dcc8e50e188a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81602) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:29:07.690094 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-c05feb4a-5efd-4055-abd6-0e376125ebc4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:29:07.690094 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-c05feb4a-5efd-4055-abd6-0e376125ebc4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:29:07.691502 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 979/1953] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 24 09:29:07 2022] DELETE /identity/v3/projects/4a2351b8835d4d4caea6dcc8e50e188a => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:29:07.696505 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7017eb9-a91a-47a7-9193-262b1e0c4a9d None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:29:07.697027 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7017eb9-a91a-47a7-9193-262b1e0c4a9d None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:29:07.697203 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7017eb9-a91a-47a7-9193-262b1e0c4a9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:29:07.718369 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-a7017eb9-a91a-47a7-9193-262b1e0c4a9d None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:29:07.719621 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-a7017eb9-a91a-47a7-9193-262b1e0c4a9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:29:07.782226 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 975/1954] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 09:29:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 09:29:07.790076 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d178e6-2c56-44e1-8f37-850f031ec923 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:29:07.790367 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d178e6-2c56-44e1-8f37-850f031ec923 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:29:07.790647 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d178e6-2c56-44e1-8f37-850f031ec923 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:29:07.810923 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-e7d178e6-2c56-44e1-8f37-850f031ec923 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:29:07.813263 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-e7d178e6-2c56-44e1-8f37-850f031ec923 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:29:07.819861 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 980/1955] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 09:29:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:29:07.824076 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7017eb9-a91a-47a7-9193-262b1e0c4a9d None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:29:07.833863 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7017eb9-a91a-47a7-9193-262b1e0c4a9d None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:29:07.839110 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f60e23-f309-4248-838f-c73c0bd5d538 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:29:07.839745 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f60e23-f309-4248-838f-c73c0bd5d538 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:29:07.840530 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f60e23-f309-4248-838f-c73c0bd5d538 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:29:07.840800 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f60e23-f309-4248-838f-c73c0bd5d538 admin admin] PATH_INFO: `/v3/domains` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:29:07.841440 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f60e23-f309-4248-838f-c73c0bd5d538 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:29:07.841739 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f60e23-f309-4248-838f-c73c0bd5d538 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:29:07.843387 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f60e23-f309-4248-838f-c73c0bd5d538 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:29:07.854857 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 976/1956] 10.222.0.16 () {58 vars in 1153 bytes} [Wed Aug 24 09:29:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:29:07.859262 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d178e6-2c56-44e1-8f37-850f031ec923 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:29:07.868713 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d178e6-2c56-44e1-8f37-850f031ec923 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:29:07.873904 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad48a170-0c1f-4b58-994d-433a0d5136ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:29:07.874501 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad48a170-0c1f-4b58-994d-433a0d5136ea admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:29:07.874779 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad48a170-0c1f-4b58-994d-433a0d5136ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:29:07.875047 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad48a170-0c1f-4b58-994d-433a0d5136ea admin admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:29:07.875962 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad48a170-0c1f-4b58-994d-433a0d5136ea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:29:07.877698 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad48a170-0c1f-4b58-994d-433a0d5136ea admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:29:07.959224 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 981/1957] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 24 09:29:07 2022] POST /identity/v3/projects => generated 385 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:29:07.963645 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f60e23-f309-4248-838f-c73c0bd5d538 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:29:07.973391 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f60e23-f309-4248-838f-c73c0bd5d538 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:29:07.985435 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:29:08.028865 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ced458-7323-40b9-9a98-099a62d1eb5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:29:08.029467 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ced458-7323-40b9-9a98-099a62d1eb5c admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:29:08.029746 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ced458-7323-40b9-9a98-099a62d1eb5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:29:08.030020 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ced458-7323-40b9-9a98-099a62d1eb5c admin admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:29:08.030649 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ced458-7323-40b9-9a98-099a62d1eb5c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:29:08.032591 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ced458-7323-40b9-9a98-099a62d1eb5c admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:29:08.123227 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 977/1958] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 24 09:29:07 2022] POST /identity/v3/users => generated 293 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:29:08.127506 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad48a170-0c1f-4b58-994d-433a0d5136ea admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:29:08.137607 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad48a170-0c1f-4b58-994d-433a0d5136ea admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:29:08.142128 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ee556c-5001-4ed3-b436-f2bcef733071 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:29:08.142721 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ee556c-5001-4ed3-b436-f2bcef733071 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:29:08.143001 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ee556c-5001-4ed3-b436-f2bcef733071 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:29:08.143272 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ee556c-5001-4ed3-b436-f2bcef733071 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:29:08.143939 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ee556c-5001-4ed3-b436-f2bcef733071 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:29:08.145578 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ee556c-5001-4ed3-b436-f2bcef733071 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:29:08.155164 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 982/1959] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 24 09:29:08 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 09:29:08.159067 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ced458-7323-40b9-9a98-099a62d1eb5c admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:29:08.169682 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ced458-7323-40b9-9a98-099a62d1eb5c admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:29:08.174367 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80a0abf-b370-48a5-bb1b-8965c0eb533a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:29:08.174962 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80a0abf-b370-48a5-bb1b-8965c0eb533a admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:29:08.175245 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80a0abf-b370-48a5-bb1b-8965c0eb533a admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:29:08.175551 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80a0abf-b370-48a5-bb1b-8965c0eb533a admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:29:08.176245 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80a0abf-b370-48a5-bb1b-8965c0eb533a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:29:08.177831 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80a0abf-b370-48a5-bb1b-8965c0eb533a admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:29:08.188940 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 978/1960] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 24 09:29:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 09:29:08.193215 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ee556c-5001-4ed3-b436-f2bcef733071 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:29:08.203895 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ee556c-5001-4ed3-b436-f2bcef733071 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:29:08.208563 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c48d99e-e284-4beb-a88a-db78a2de16f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:29:08.209211 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c48d99e-e284-4beb-a88a-db78a2de16f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:29:08.209489 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c48d99e-e284-4beb-a88a-db78a2de16f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:29:08.209760 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c48d99e-e284-4beb-a88a-db78a2de16f8 admin admin] PATH_INFO: `/v3/projects/e88eb143521d4d0482e4bcdba8488a88/users/4b901e0f8b4a41d79d1e09d9da276022/roles/136cdf78e00a426997dd4e37faff489b` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:29:08.226431 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c48d99e-e284-4beb-a88a-db78a2de16f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e88eb143521d4d0482e4bcdba8488a88, user_id=4b901e0f8b4a41d79d1e09d9da276022, role_id=136cdf78e00a426997dd4e37faff489b)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:29:08.230339 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c48d99e-e284-4beb-a88a-db78a2de16f8 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:29:08.335471 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 983/1961] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 24 09:29:08 2022] PUT /identity/v3/projects/e88eb143521d4d0482e4bcdba8488a88/users/4b901e0f8b4a41d79d1e09d9da276022/roles/136cdf78e00a426997dd4e37faff489b => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:29:08.347739 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16479edb-4edf-4c44-b892-b3e5ef143675 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:29:08.348058 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16479edb-4edf-4c44-b892-b3e5ef143675 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:29:08.348334 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16479edb-4edf-4c44-b892-b3e5ef143675 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:29:08.376447 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-16479edb-4edf-4c44-b892-b3e5ef143675 None None] MFA Rules not processed for user `4b901e0f8b4a41d79d1e09d9da276022`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:29:08.377596 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-16479edb-4edf-4c44-b892-b3e5ef143675 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:29:08.440827 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 979/1962] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 09:29:08 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:29:08.452511 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c48d99e-e284-4beb-a88a-db78a2de16f8 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:29:08.454246 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-4c48d99e-e284-4beb-a88a-db78a2de16f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:29:08.500309 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:29:08.511175 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c48d99e-e284-4beb-a88a-db78a2de16f8 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:29:08.515535 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cae33e-8451-4008-80c1-839afb7ac0d6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c32b83577c46488906fe409a790b24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:29:08.516185 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cae33e-8451-4008-80c1-839afb7ac0d6 service cinder] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:29:08.516470 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cae33e-8451-4008-80c1-839afb7ac0d6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:29:08.516742 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cae33e-8451-4008-80c1-839afb7ac0d6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:29:08.524799 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cae33e-8451-4008-80c1-839afb7ac0d6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:29:08.526220 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cae33e-8451-4008-80c1-839afb7ac0d6 service cinder] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:29:08.535970 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 984/1963] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:29:08 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:29:08.555273 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16479edb-4edf-4c44-b892-b3e5ef143675 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:29:08.617854 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16479edb-4edf-4c44-b892-b3e5ef143675 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:29:08.622508 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd42f9d3-6d31-407c-b656-da05051f8136 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:29:08.623138 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd42f9d3-6d31-407c-b656-da05051f8136 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:29:08.623418 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd42f9d3-6d31-407c-b656-da05051f8136 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:29:08.623728 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd42f9d3-6d31-407c-b656-da05051f8136 admin admin] PATH_INFO: `/v3/users/4b901e0f8b4a41d79d1e09d9da276022` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:29:08.624853 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd42f9d3-6d31-407c-b656-da05051f8136 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4b901e0f8b4a41d79d1e09d9da276022)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:29:08.626948 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd42f9d3-6d31-407c-b656-da05051f8136 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:29:08.736962 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:29:08.824429 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-dd42f9d3-6d31-407c-b656-da05051f8136 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4b901e0f8b4a41d79d1e09d9da276022'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:29:08.825146 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-dd42f9d3-6d31-407c-b656-da05051f8136 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4b901e0f8b4a41d79d1e09d9da276022'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:29:08.839210 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-dd42f9d3-6d31-407c-b656-da05051f8136 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4b901e0f8b4a41d79d1e09d9da276022'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:29:08.846055 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-dd42f9d3-6d31-407c-b656-da05051f8136 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4b901e0f8b4a41d79d1e09d9da276022'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:29:08.906502 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-dd42f9d3-6d31-407c-b656-da05051f8136 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4b901e0f8b4a41d79d1e09d9da276022'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:29:08.913071 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 980/1964] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 24 09:29:08 2022] DELETE /identity/v3/users/4b901e0f8b4a41d79d1e09d9da276022 => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:29:08.917537 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cd9411-fe87-41ec-a14e-c107f1f7078b None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:29:08.917817 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cd9411-fe87-41ec-a14e-c107f1f7078b None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:29:08.918662 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cd9411-fe87-41ec-a14e-c107f1f7078b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:29:08.940154 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-a0cd9411-fe87-41ec-a14e-c107f1f7078b None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:29:08.940329 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-a0cd9411-fe87-41ec-a14e-c107f1f7078b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:29:09.014278 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 985/1965] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 09:29:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:29:09.023879 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd42f9d3-6d31-407c-b656-da05051f8136 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:29:09.025279 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-dd42f9d3-6d31-407c-b656-da05051f8136 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:29:09.083352 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd42f9d3-6d31-407c-b656-da05051f8136 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:29:09.087256 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6c5e67-af8c-4131-8a27-857aaa88ad09 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e5260e798404e8bb12b25ca8ea97052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:29:09.087940 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6c5e67-af8c-4131-8a27-857aaa88ad09 service neutron] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:29:09.089408 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6c5e67-af8c-4131-8a27-857aaa88ad09 service neutron] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:29:09.089408 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6c5e67-af8c-4131-8a27-857aaa88ad09 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:29:09.098112 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6c5e67-af8c-4131-8a27-857aaa88ad09 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:29:09.099650 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6c5e67-af8c-4131-8a27-857aaa88ad09 service neutron] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:29:09.112420 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 981/1966] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:29:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:29:09.828873 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cd9411-fe87-41ec-a14e-c107f1f7078b None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:29:09.830559 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-a0cd9411-fe87-41ec-a14e-c107f1f7078b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:29:09.843500 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cd9411-fe87-41ec-a14e-c107f1f7078b None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:29:09.848950 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d8a130-9220-4143-ad71-77c48cf6096c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:29:09.849593 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d8a130-9220-4143-ad71-77c48cf6096c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:29:09.849873 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d8a130-9220-4143-ad71-77c48cf6096c admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:29:09.850275 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d8a130-9220-4143-ad71-77c48cf6096c admin admin] PATH_INFO: `/v3/projects/e88eb143521d4d0482e4bcdba8488a88` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:29:09.851720 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d8a130-9220-4143-ad71-77c48cf6096c admin admin] RBAC: Authorizing `identity:delete_project(project_id=e88eb143521d4d0482e4bcdba8488a88)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:29:09.853777 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d8a130-9220-4143-ad71-77c48cf6096c admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:29:09.987576 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-39d8a130-9220-4143-ad71-77c48cf6096c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e88eb143521d4d0482e4bcdba8488a88', 'request_id': 'req-39d8a130-9220-4143-ad71-77c48cf6096c'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:29:09.998825 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-39d8a130-9220-4143-ad71-77c48cf6096c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e88eb143521d4d0482e4bcdba8488a88', 'request_id': 'req-39d8a130-9220-4143-ad71-77c48cf6096c'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:29:10.007249 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-39d8a130-9220-4143-ad71-77c48cf6096c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e88eb143521d4d0482e4bcdba8488a88', 'request_id': 'req-39d8a130-9220-4143-ad71-77c48cf6096c'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:29:10.079077 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-39d8a130-9220-4143-ad71-77c48cf6096c admin admin] The token cache is being invalidate because project e88eb143521d4d0482e4bcdba8488a88 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81602) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:29:10.079501 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-39d8a130-9220-4143-ad71-77c48cf6096c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:29:10.080404 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-39d8a130-9220-4143-ad71-77c48cf6096c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:29:10.088915 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 986/1967] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 24 09:29:09 2022] DELETE /identity/v3/projects/e88eb143521d4d0482e4bcdba8488a88 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:29:10.090358 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6c5e67-af8c-4131-8a27-857aaa88ad09 service neutron] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:29:10.092149 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-2a6c5e67-af8c-4131-8a27-857aaa88ad09 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:29:10.094072 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f53afdc-f529-4ece-b1f7-65586db5b504 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:29:10.096183 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f53afdc-f529-4ece-b1f7-65586db5b504 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:29:10.096183 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f53afdc-f529-4ece-b1f7-65586db5b504 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:29:10.109671 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6c5e67-af8c-4131-8a27-857aaa88ad09 service neutron] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:29:10.144561 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-4f53afdc-f529-4ece-b1f7-65586db5b504 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:29:10.146012 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-4f53afdc-f529-4ece-b1f7-65586db5b504 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:29:10.249307 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623ba286-ba1f-404a-a872-ba83414a0e91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:29:10.252140 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623ba286-ba1f-404a-a872-ba83414a0e91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:29:10.252449 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623ba286-ba1f-404a-a872-ba83414a0e91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:29:10.254312 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623ba286-ba1f-404a-a872-ba83414a0e91 admin admin] PATH_INFO: `/v3/users/35e4f58d358e48fe83677a3df6a083b1` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:29:10.259604 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623ba286-ba1f-404a-a872-ba83414a0e91 admin admin] RBAC: Authorizing `identity:delete_user(user_id=35e4f58d358e48fe83677a3df6a083b1)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:29:10.265949 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623ba286-ba1f-404a-a872-ba83414a0e91 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:29:10.307650 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 987/1968] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 09:29:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:29:10.316540 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90721309-a14f-4e9f-845b-482acbe0ca65 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:29:10.316938 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90721309-a14f-4e9f-845b-482acbe0ca65 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:29:10.317304 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90721309-a14f-4e9f-845b-482acbe0ca65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:29:10.342105 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-90721309-a14f-4e9f-845b-482acbe0ca65 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:29:10.347973 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-90721309-a14f-4e9f-845b-482acbe0ca65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:29:10.365394 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 988/1969] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 09:29:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:29:10.366888 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90721309-a14f-4e9f-845b-482acbe0ca65 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:29:10.382822 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:29:10.390445 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90721309-a14f-4e9f-845b-482acbe0ca65 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:29:10.397021 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab02739-503a-4b42-9e27-8bc6d1b89cef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:29:10.400982 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab02739-503a-4b42-9e27-8bc6d1b89cef admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:29:10.401267 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab02739-503a-4b42-9e27-8bc6d1b89cef admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:29:10.401542 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab02739-503a-4b42-9e27-8bc6d1b89cef admin admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:29:10.402209 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab02739-503a-4b42-9e27-8bc6d1b89cef admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:29:10.403069 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab02739-503a-4b42-9e27-8bc6d1b89cef admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:29:10.411823 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab02739-503a-4b42-9e27-8bc6d1b89cef admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:29:10.437189 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 989/1970] 10.222.0.16 () {58 vars in 1153 bytes} [Wed Aug 24 09:29:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:29:10.443196 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab02739-503a-4b42-9e27-8bc6d1b89cef admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:29:10.459106 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab02739-503a-4b42-9e27-8bc6d1b89cef admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:29:10.466074 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6ba93d-3450-4b1d-a828-166f40098958 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:29:10.466796 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6ba93d-3450-4b1d-a828-166f40098958 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:29:10.467204 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6ba93d-3450-4b1d-a828-166f40098958 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:29:10.467551 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6ba93d-3450-4b1d-a828-166f40098958 admin admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:29:10.468452 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6ba93d-3450-4b1d-a828-166f40098958 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:29:10.471920 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6ba93d-3450-4b1d-a828-166f40098958 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:29:10.666145 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 990/1971] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 24 09:29:10 2022] POST /identity/v3/projects => generated 373 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:29:10.669767 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6ba93d-3450-4b1d-a828-166f40098958 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:29:10.743213 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6ba93d-3450-4b1d-a828-166f40098958 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:29:10.746222 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-623ba286-ba1f-404a-a872-ba83414a0e91 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '35e4f58d358e48fe83677a3df6a083b1'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:29:10.746879 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-623ba286-ba1f-404a-a872-ba83414a0e91 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '35e4f58d358e48fe83677a3df6a083b1'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:29:10.748546 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a18b4ea-677b-4b39-89f1-e4bad23ec238 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:29:10.749033 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:29:10.749564 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a18b4ea-677b-4b39-89f1-e4bad23ec238 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:29:10.749916 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a18b4ea-677b-4b39-89f1-e4bad23ec238 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:29:10.750189 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a18b4ea-677b-4b39-89f1-e4bad23ec238 admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:29:10.751048 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a18b4ea-677b-4b39-89f1-e4bad23ec238 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:29:10.753014 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a18b4ea-677b-4b39-89f1-e4bad23ec238 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:29:10.767258 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-623ba286-ba1f-404a-a872-ba83414a0e91 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '35e4f58d358e48fe83677a3df6a083b1'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:29:10.768394 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-623ba286-ba1f-404a-a872-ba83414a0e91 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '35e4f58d358e48fe83677a3df6a083b1'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:29:10.848094 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 991/1972] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 24 09:29:10 2022] POST /identity/v3/users => generated 287 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 09:29:10.849699 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a18b4ea-677b-4b39-89f1-e4bad23ec238 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:29:10.860550 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a18b4ea-677b-4b39-89f1-e4bad23ec238 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:29:10.865712 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e627b93-e9a4-4010-b54e-b7359754201f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:29:10.866318 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e627b93-e9a4-4010-b54e-b7359754201f admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:29:10.866596 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e627b93-e9a4-4010-b54e-b7359754201f admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:29:10.866863 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e627b93-e9a4-4010-b54e-b7359754201f admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:29:10.867214 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-623ba286-ba1f-404a-a872-ba83414a0e91 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '35e4f58d358e48fe83677a3df6a083b1'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:29:10.867507 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e627b93-e9a4-4010-b54e-b7359754201f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:29:10.869176 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e627b93-e9a4-4010-b54e-b7359754201f admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:29:10.880338 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 982/1973] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 24 09:29:10 2022] DELETE /identity/v3/users/35e4f58d358e48fe83677a3df6a083b1 => generated 0 bytes in 788 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 09:29:10.881217 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 992/1974] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 24 09:29:10 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:29:10.882027 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbe2577-866f-41c5-9e25-2759e8f4160e None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:29:10.882337 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbe2577-866f-41c5-9e25-2759e8f4160e None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:29:10.882598 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbe2577-866f-41c5-9e25-2759e8f4160e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:29:10.885202 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e627b93-e9a4-4010-b54e-b7359754201f admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:29:10.887019 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-3e627b93-e9a4-4010-b54e-b7359754201f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:29:10.901112 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e627b93-e9a4-4010-b54e-b7359754201f admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:29:10.905563 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-7bbe2577-866f-41c5-9e25-2759e8f4160e None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:29:10.906800 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-7bbe2577-866f-41c5-9e25-2759e8f4160e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:29:10.961109 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3efd0c2-e571-4d88-9a58-42101eb6720a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:29:10.962220 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3efd0c2-e571-4d88-9a58-42101eb6720a admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:29:10.962511 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3efd0c2-e571-4d88-9a58-42101eb6720a admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:29:10.962795 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3efd0c2-e571-4d88-9a58-42101eb6720a admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:29:10.963446 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3efd0c2-e571-4d88-9a58-42101eb6720a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:29:10.965605 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3efd0c2-e571-4d88-9a58-42101eb6720a admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:29:10.982792 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 993/1975] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 24 09:29:10 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:29:10.987033 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3efd0c2-e571-4d88-9a58-42101eb6720a admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:29:10.993203 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 983/1976] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 09:29:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:29:11.004848 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3efd0c2-e571-4d88-9a58-42101eb6720a admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:29:11.005390 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbe2577-866f-41c5-9e25-2759e8f4160e None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:29:11.006980 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-7bbe2577-866f-41c5-9e25-2759e8f4160e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:29:11.010549 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bdf25d-b8f2-4720-9abd-789d197361e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:29:11.014059 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bdf25d-b8f2-4720-9abd-789d197361e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:29:11.014059 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bdf25d-b8f2-4720-9abd-789d197361e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:29:11.014059 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bdf25d-b8f2-4720-9abd-789d197361e9 admin admin] PATH_INFO: `/v3/projects/6204c9e825f841aab905eac66e6a4a1e/users/b3cd284afb854b1e987e807127752925/roles/136cdf78e00a426997dd4e37faff489b` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:29:11.039790 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bdf25d-b8f2-4720-9abd-789d197361e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6204c9e825f841aab905eac66e6a4a1e, user_id=b3cd284afb854b1e987e807127752925, role_id=136cdf78e00a426997dd4e37faff489b)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:29:11.042644 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bdf25d-b8f2-4720-9abd-789d197361e9 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:29:11.100388 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbe2577-866f-41c5-9e25-2759e8f4160e None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:29:11.104170 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e2431a-5efa-4904-90f5-2df538a8a0e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e5260e798404e8bb12b25ca8ea97052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:29:11.104896 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e2431a-5efa-4904-90f5-2df538a8a0e5 service neutron] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:29:11.105207 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e2431a-5efa-4904-90f5-2df538a8a0e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:29:11.106114 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e2431a-5efa-4904-90f5-2df538a8a0e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:29:11.124082 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e2431a-5efa-4904-90f5-2df538a8a0e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:29:11.125547 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e2431a-5efa-4904-90f5-2df538a8a0e5 service neutron] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:29:11.136329 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 984/1977] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:29:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 09:29:11.148539 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 994/1978] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 24 09:29:10 2022] PUT /identity/v3/projects/6204c9e825f841aab905eac66e6a4a1e/users/b3cd284afb854b1e987e807127752925/roles/136cdf78e00a426997dd4e37faff489b => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:29:11.153825 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32988576-271e-403c-8bbb-e4c3cb6d2d2d None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:29:11.154300 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32988576-271e-403c-8bbb-e4c3cb6d2d2d None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:29:11.154745 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32988576-271e-403c-8bbb-e4c3cb6d2d2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:29:11.185037 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-32988576-271e-403c-8bbb-e4c3cb6d2d2d None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:29:11.185037 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-32988576-271e-403c-8bbb-e4c3cb6d2d2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:29:11.295915 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 985/1979] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 09:29:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:29:11.306728 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bdf25d-b8f2-4720-9abd-789d197361e9 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:29:11.365375 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bdf25d-b8f2-4720-9abd-789d197361e9 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:29:11.374763 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8089b9ec-2b05-4b93-abee-1a327842068f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e5260e798404e8bb12b25ca8ea97052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:29:11.374763 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8089b9ec-2b05-4b93-abee-1a327842068f service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:29:11.374763 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8089b9ec-2b05-4b93-abee-1a327842068f service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:29:11.374763 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8089b9ec-2b05-4b93-abee-1a327842068f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:29:11.380700 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8089b9ec-2b05-4b93-abee-1a327842068f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:29:11.382144 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8089b9ec-2b05-4b93-abee-1a327842068f service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:29:11.392103 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 995/1980] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:29:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:29:11.697688 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32988576-271e-403c-8bbb-e4c3cb6d2d2d None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:29:11.698988 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-32988576-271e-403c-8bbb-e4c3cb6d2d2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:29:11.714498 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32988576-271e-403c-8bbb-e4c3cb6d2d2d None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:29:11.720209 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88d8305-91d7-460d-affc-506a6c40a5e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:29:11.720854 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88d8305-91d7-460d-affc-506a6c40a5e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:29:11.721141 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88d8305-91d7-460d-affc-506a6c40a5e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:29:11.721414 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88d8305-91d7-460d-affc-506a6c40a5e3 admin admin] PATH_INFO: `/v3/projects/786299295cf944b2b4fc909b9dbe72a4` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:29:11.722650 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88d8305-91d7-460d-affc-506a6c40a5e3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=786299295cf944b2b4fc909b9dbe72a4)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:29:11.724681 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88d8305-91d7-460d-affc-506a6c40a5e3 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:29:11.839298 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-c88d8305-91d7-460d-affc-506a6c40a5e3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '786299295cf944b2b4fc909b9dbe72a4', 'request_id': 'req-c88d8305-91d7-460d-affc-506a6c40a5e3'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:29:11.846224 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-c88d8305-91d7-460d-affc-506a6c40a5e3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '786299295cf944b2b4fc909b9dbe72a4', 'request_id': 'req-c88d8305-91d7-460d-affc-506a6c40a5e3'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:29:11.853187 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-c88d8305-91d7-460d-affc-506a6c40a5e3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '786299295cf944b2b4fc909b9dbe72a4', 'request_id': 'req-c88d8305-91d7-460d-affc-506a6c40a5e3'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:29:11.936019 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-c88d8305-91d7-460d-affc-506a6c40a5e3 admin admin] The token cache is being invalidate because project 786299295cf944b2b4fc909b9dbe72a4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:29:11.936727 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-c88d8305-91d7-460d-affc-506a6c40a5e3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:29:11.937173 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-c88d8305-91d7-460d-affc-506a6c40a5e3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:29:11.941556 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 986/1981] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 24 09:29:11 2022] DELETE /identity/v3/projects/786299295cf944b2b4fc909b9dbe72a4 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:29:11.943274 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8089b9ec-2b05-4b93-abee-1a327842068f service neutron] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:29:11.953943 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8089b9ec-2b05-4b93-abee-1a327842068f service neutron] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:29:12.039286 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b7cb44-e77d-4adf-a449-47332a8c4a83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:29:12.040046 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b7cb44-e77d-4adf-a449-47332a8c4a83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:29:12.040327 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b7cb44-e77d-4adf-a449-47332a8c4a83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:29:12.040601 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b7cb44-e77d-4adf-a449-47332a8c4a83 admin admin] PATH_INFO: `/v3/users/59833b44efa644f1b030101f63c6d939` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:29:12.042160 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b7cb44-e77d-4adf-a449-47332a8c4a83 admin admin] RBAC: Authorizing `identity:delete_user(user_id=59833b44efa644f1b030101f63c6d939)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:29:12.044668 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b7cb44-e77d-4adf-a449-47332a8c4a83 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:29:12.063819 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:29:12.327870 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-15b7cb44-e77d-4adf-a449-47332a8c4a83 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '59833b44efa644f1b030101f63c6d939'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:29:12.328722 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-15b7cb44-e77d-4adf-a449-47332a8c4a83 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '59833b44efa644f1b030101f63c6d939'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:29:12.415169 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-15b7cb44-e77d-4adf-a449-47332a8c4a83 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '59833b44efa644f1b030101f63c6d939'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:29:12.415829 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-15b7cb44-e77d-4adf-a449-47332a8c4a83 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '59833b44efa644f1b030101f63c6d939'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:29:12.425112 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-15b7cb44-e77d-4adf-a449-47332a8c4a83 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '59833b44efa644f1b030101f63c6d939'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:29:12.438519 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:29:12.444109 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 996/1982] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 24 09:29:11 2022] DELETE /identity/v3/users/59833b44efa644f1b030101f63c6d939 => generated 0 bytes in 501 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:29:12.758404 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd23dcf-d0da-4cea-80df-3d47364d38d9 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:29:12.758969 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd23dcf-d0da-4cea-80df-3d47364d38d9 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:29:12.759468 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd23dcf-d0da-4cea-80df-3d47364d38d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:29:12.792967 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-3fd23dcf-d0da-4cea-80df-3d47364d38d9 None None] MFA Rules not processed for user `b3cd284afb854b1e987e807127752925`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:29:12.794742 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-3fd23dcf-d0da-4cea-80df-3d47364d38d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:29:12.861374 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 987/1983] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 09:29:12 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:29:12.872980 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b7cb44-e77d-4adf-a449-47332a8c4a83 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:29:12.874332 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-15b7cb44-e77d-4adf-a449-47332a8c4a83 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:29:12.896455 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd23dcf-d0da-4cea-80df-3d47364d38d9 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:29:12.898496 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-3fd23dcf-d0da-4cea-80df-3d47364d38d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:29:12.940148 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b7cb44-e77d-4adf-a449-47332a8c4a83 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:29:12.943893 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bebc11-83fa-4c67-836a-bcf20d1e8d1f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c32b83577c46488906fe409a790b24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:29:12.944502 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bebc11-83fa-4c67-836a-bcf20d1e8d1f service cinder] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:29:12.944777 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bebc11-83fa-4c67-836a-bcf20d1e8d1f service cinder] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:29:12.945116 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bebc11-83fa-4c67-836a-bcf20d1e8d1f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:29:12.953672 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bebc11-83fa-4c67-836a-bcf20d1e8d1f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:29:12.955108 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bebc11-83fa-4c67-836a-bcf20d1e8d1f service cinder] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:29:12.964515 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 997/1984] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:29:12 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 09:29:12.972206 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd23dcf-d0da-4cea-80df-3d47364d38d9 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:29:12.976959 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7031bf-c95e-4ded-82ea-77ba71479428 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:29:12.977602 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7031bf-c95e-4ded-82ea-77ba71479428 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:29:12.977885 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7031bf-c95e-4ded-82ea-77ba71479428 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:29:12.978163 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7031bf-c95e-4ded-82ea-77ba71479428 admin admin] PATH_INFO: `/v3/projects/40f66d2f21354a7b87c731fe40cfee23` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:29:12.979289 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7031bf-c95e-4ded-82ea-77ba71479428 admin admin] RBAC: Authorizing `identity:delete_project(project_id=40f66d2f21354a7b87c731fe40cfee23)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:29:12.981865 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7031bf-c95e-4ded-82ea-77ba71479428 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:29:13.089538 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-cb7031bf-c95e-4ded-82ea-77ba71479428 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '40f66d2f21354a7b87c731fe40cfee23', 'request_id': 'req-cb7031bf-c95e-4ded-82ea-77ba71479428'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:29:13.097064 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-cb7031bf-c95e-4ded-82ea-77ba71479428 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '40f66d2f21354a7b87c731fe40cfee23', 'request_id': 'req-cb7031bf-c95e-4ded-82ea-77ba71479428'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:29:13.102924 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-cb7031bf-c95e-4ded-82ea-77ba71479428 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '40f66d2f21354a7b87c731fe40cfee23', 'request_id': 'req-cb7031bf-c95e-4ded-82ea-77ba71479428'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:29:13.277394 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-cb7031bf-c95e-4ded-82ea-77ba71479428 admin admin] The token cache is being invalidate because project 40f66d2f21354a7b87c731fe40cfee23 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:29:13.277620 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-cb7031bf-c95e-4ded-82ea-77ba71479428 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:29:13.278293 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-cb7031bf-c95e-4ded-82ea-77ba71479428 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:29:13.279873 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 988/1985] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 24 09:29:12 2022] DELETE /identity/v3/projects/40f66d2f21354a7b87c731fe40cfee23 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 09:29:13.284767 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a92afef-8882-47ba-b873-620047106814 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:29:13.285122 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a92afef-8882-47ba-b873-620047106814 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:29:13.285447 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a92afef-8882-47ba-b873-620047106814 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:29:13.305499 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-0a92afef-8882-47ba-b873-620047106814 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:29:13.306725 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-0a92afef-8882-47ba-b873-620047106814 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:29:13.366183 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 998/1986] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 09:29:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:29:13.375880 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9cef89-add3-4ae0-a43a-8d0da4c7c5ab None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:29:13.378994 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9cef89-add3-4ae0-a43a-8d0da4c7c5ab None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:29:13.378994 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9cef89-add3-4ae0-a43a-8d0da4c7c5ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:29:13.396248 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-4c9cef89-add3-4ae0-a43a-8d0da4c7c5ab None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:29:13.397459 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-4c9cef89-add3-4ae0-a43a-8d0da4c7c5ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:29:13.405472 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 989/1987] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 09:29:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:29:13.410011 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a92afef-8882-47ba-b873-620047106814 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:29:13.420358 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a92afef-8882-47ba-b873-620047106814 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:29:13.424997 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149beae7-071d-499e-815e-78cdc45b452c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:29:13.434537 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149beae7-071d-499e-815e-78cdc45b452c admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:29:13.434537 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149beae7-071d-499e-815e-78cdc45b452c admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:29:13.434537 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149beae7-071d-499e-815e-78cdc45b452c admin admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:29:13.434537 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:29:13.434537 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149beae7-071d-499e-815e-78cdc45b452c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:29:13.434537 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149beae7-071d-499e-815e-78cdc45b452c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:29:13.435326 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149beae7-071d-499e-815e-78cdc45b452c admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:29:13.444481 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 999/1988] 10.222.0.16 () {58 vars in 1153 bytes} [Wed Aug 24 09:29:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:29:13.449143 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9cef89-add3-4ae0-a43a-8d0da4c7c5ab None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:29:13.459034 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9cef89-add3-4ae0-a43a-8d0da4c7c5ab None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:29:13.464516 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5ef49d-7228-4fef-83cd-e7ff9693c42a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:29:13.465777 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5ef49d-7228-4fef-83cd-e7ff9693c42a admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:29:13.466054 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5ef49d-7228-4fef-83cd-e7ff9693c42a admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:29:13.466353 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5ef49d-7228-4fef-83cd-e7ff9693c42a admin admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:29:13.466982 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5ef49d-7228-4fef-83cd-e7ff9693c42a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:29:13.468934 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5ef49d-7228-4fef-83cd-e7ff9693c42a admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:29:13.548379 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 990/1989] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 24 09:29:13 2022] POST /identity/v3/projects => generated 375 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 09:29:13.553283 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149beae7-071d-499e-815e-78cdc45b452c admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:29:13.566535 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149beae7-071d-499e-815e-78cdc45b452c admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:29:13.623698 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9c91f8-9002-4ec7-bdb1-94d0b621dd67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:29:13.624497 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9c91f8-9002-4ec7-bdb1-94d0b621dd67 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:29:13.624783 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9c91f8-9002-4ec7-bdb1-94d0b621dd67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:29:13.625076 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9c91f8-9002-4ec7-bdb1-94d0b621dd67 admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:29:13.625696 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9c91f8-9002-4ec7-bdb1-94d0b621dd67 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:29:13.627620 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9c91f8-9002-4ec7-bdb1-94d0b621dd67 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:29:13.818533 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1000/1990] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 24 09:29:13 2022] POST /identity/v3/users => generated 288 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:29:13.819800 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5ef49d-7228-4fef-83cd-e7ff9693c42a admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:29:13.843746 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5ef49d-7228-4fef-83cd-e7ff9693c42a admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:29:13.851684 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d7bfb9-f3c7-4336-b07a-2637e09264fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:29:13.852582 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d7bfb9-f3c7-4336-b07a-2637e09264fb admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:29:13.852834 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d7bfb9-f3c7-4336-b07a-2637e09264fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:29:13.853111 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d7bfb9-f3c7-4336-b07a-2637e09264fb admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:29:13.853761 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d7bfb9-f3c7-4336-b07a-2637e09264fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:29:13.855438 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d7bfb9-f3c7-4336-b07a-2637e09264fb admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:29:13.865722 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 991/1991] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 24 09:29:13 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:29:13.869908 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9c91f8-9002-4ec7-bdb1-94d0b621dd67 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:29:13.887398 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9c91f8-9002-4ec7-bdb1-94d0b621dd67 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:29:13.892739 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89691b58-5f3f-453e-8580-77fc37b2d65c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:29:13.893867 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89691b58-5f3f-453e-8580-77fc37b2d65c admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:29:13.894215 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89691b58-5f3f-453e-8580-77fc37b2d65c admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:29:13.894649 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89691b58-5f3f-453e-8580-77fc37b2d65c admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:29:13.895365 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89691b58-5f3f-453e-8580-77fc37b2d65c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:29:13.897172 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89691b58-5f3f-453e-8580-77fc37b2d65c admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:29:13.908834 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1001/1992] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 24 09:29:13 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:29:13.913243 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d7bfb9-f3c7-4336-b07a-2637e09264fb admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:29:13.923373 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d7bfb9-f3c7-4336-b07a-2637e09264fb admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:29:13.928169 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53433baf-6ecc-4432-b4f5-4e965250ea51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:29:13.928962 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53433baf-6ecc-4432-b4f5-4e965250ea51 admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:29:13.929317 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53433baf-6ecc-4432-b4f5-4e965250ea51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:29:13.929658 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53433baf-6ecc-4432-b4f5-4e965250ea51 admin admin] PATH_INFO: `/v3/projects/6107a70b787c455d8fa96b065ba42cb7/users/1cd6bfe7f1fa4487afe63292fb1f02f8/roles/136cdf78e00a426997dd4e37faff489b` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:29:13.947626 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53433baf-6ecc-4432-b4f5-4e965250ea51 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6107a70b787c455d8fa96b065ba42cb7, user_id=1cd6bfe7f1fa4487afe63292fb1f02f8, role_id=136cdf78e00a426997dd4e37faff489b)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:29:13.951043 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53433baf-6ecc-4432-b4f5-4e965250ea51 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:29:14.032415 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 992/1993] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 24 09:29:13 2022] PUT /identity/v3/projects/6107a70b787c455d8fa96b065ba42cb7/users/1cd6bfe7f1fa4487afe63292fb1f02f8/roles/136cdf78e00a426997dd4e37faff489b => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:29:14.042778 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c8143b-9ccb-4743-9e92-1d8e100f2632 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:29:14.043785 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c8143b-9ccb-4743-9e92-1d8e100f2632 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:29:14.043874 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c8143b-9ccb-4743-9e92-1d8e100f2632 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:29:14.071422 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-f2c8143b-9ccb-4743-9e92-1d8e100f2632 None None] MFA Rules not processed for user `1cd6bfe7f1fa4487afe63292fb1f02f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:29:14.072588 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-f2c8143b-9ccb-4743-9e92-1d8e100f2632 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:29:14.137467 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1002/1994] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 09:29:14 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:29:14.147472 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53433baf-6ecc-4432-b4f5-4e965250ea51 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:29:14.198775 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:29:14.221643 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53433baf-6ecc-4432-b4f5-4e965250ea51 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:29:14.226336 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f8e2dc-e485-4eee-ac90-7461d94b0a01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:29:14.226948 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f8e2dc-e485-4eee-ac90-7461d94b0a01 admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:29:14.227235 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f8e2dc-e485-4eee-ac90-7461d94b0a01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:29:14.227504 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f8e2dc-e485-4eee-ac90-7461d94b0a01 admin admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:29:14.228155 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f8e2dc-e485-4eee-ac90-7461d94b0a01 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:29:14.230049 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f8e2dc-e485-4eee-ac90-7461d94b0a01 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:29:14.319165 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 993/1995] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 24 09:29:14 2022] POST /identity/v3/projects => generated 377 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:29:14.324150 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c8143b-9ccb-4743-9e92-1d8e100f2632 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:29:14.335355 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c8143b-9ccb-4743-9e92-1d8e100f2632 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:29:14.391163 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d77af6a-8b86-4979-bdcb-f940ed027873 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:29:14.391799 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d77af6a-8b86-4979-bdcb-f940ed027873 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:29:14.392082 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d77af6a-8b86-4979-bdcb-f940ed027873 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:29:14.392353 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d77af6a-8b86-4979-bdcb-f940ed027873 admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:29:14.392965 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d77af6a-8b86-4979-bdcb-f940ed027873 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:29:14.394880 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d77af6a-8b86-4979-bdcb-f940ed027873 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:29:14.491930 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1003/1996] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 24 09:29:14 2022] POST /identity/v3/users => generated 289 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:29:14.493772 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f8e2dc-e485-4eee-ac90-7461d94b0a01 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:29:14.505218 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f8e2dc-e485-4eee-ac90-7461d94b0a01 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:29:14.509815 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ea3d30-0037-4a45-9e55-f5ff7c8103a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:29:14.510427 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ea3d30-0037-4a45-9e55-f5ff7c8103a3 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:29:14.513817 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ea3d30-0037-4a45-9e55-f5ff7c8103a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:29:14.513817 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ea3d30-0037-4a45-9e55-f5ff7c8103a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:29:14.513817 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ea3d30-0037-4a45-9e55-f5ff7c8103a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:29:14.513817 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ea3d30-0037-4a45-9e55-f5ff7c8103a3 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:29:14.524051 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 994/1997] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 24 09:29:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:29:14.528182 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d77af6a-8b86-4979-bdcb-f940ed027873 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:29:14.537929 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d77af6a-8b86-4979-bdcb-f940ed027873 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:29:14.542454 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a0f5e1-b361-4d97-afec-d45d60ea4116 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:29:14.543099 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a0f5e1-b361-4d97-afec-d45d60ea4116 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:29:14.543375 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a0f5e1-b361-4d97-afec-d45d60ea4116 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:29:14.543700 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a0f5e1-b361-4d97-afec-d45d60ea4116 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:29:14.544338 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a0f5e1-b361-4d97-afec-d45d60ea4116 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:29:14.546013 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a0f5e1-b361-4d97-afec-d45d60ea4116 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:29:14.555779 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1004/1998] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 24 09:29:14 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:29:14.560317 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ea3d30-0037-4a45-9e55-f5ff7c8103a3 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:29:14.570056 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ea3d30-0037-4a45-9e55-f5ff7c8103a3 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:29:14.575334 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287f0639-ccad-4e6f-bc13-fa217a97a345 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:29:14.575871 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:29:14.575995 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287f0639-ccad-4e6f-bc13-fa217a97a345 admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:29:14.576249 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287f0639-ccad-4e6f-bc13-fa217a97a345 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:29:14.576522 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287f0639-ccad-4e6f-bc13-fa217a97a345 admin admin] PATH_INFO: `/v3/projects/b69eb735ad95443095fe7f0ffd6b9c06/users/cfccf1385b684eb9b2d40c1cb3c2e54c/roles/136cdf78e00a426997dd4e37faff489b` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:29:14.590960 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287f0639-ccad-4e6f-bc13-fa217a97a345 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b69eb735ad95443095fe7f0ffd6b9c06, user_id=cfccf1385b684eb9b2d40c1cb3c2e54c, role_id=136cdf78e00a426997dd4e37faff489b)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:29:14.593638 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287f0639-ccad-4e6f-bc13-fa217a97a345 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:29:14.683759 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 995/1999] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 24 09:29:14 2022] PUT /identity/v3/projects/b69eb735ad95443095fe7f0ffd6b9c06/users/cfccf1385b684eb9b2d40c1cb3c2e54c/roles/136cdf78e00a426997dd4e37faff489b => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:29:14.693766 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e94b39-5e4f-459c-99e7-bc0caead414d None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:29:14.698433 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e94b39-5e4f-459c-99e7-bc0caead414d None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:29:14.698433 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e94b39-5e4f-459c-99e7-bc0caead414d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:29:14.723862 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-13e94b39-5e4f-459c-99e7-bc0caead414d None None] MFA Rules not processed for user `cfccf1385b684eb9b2d40c1cb3c2e54c`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:29:14.725607 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-13e94b39-5e4f-459c-99e7-bc0caead414d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:29:14.780884 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:DELETE=6,keystone:SELECT=89,keystone:INSERT=5 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:29:14.789334 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1005/2000] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 09:29:14 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:29:14.793729 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287f0639-ccad-4e6f-bc13-fa217a97a345 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:29:14.854594 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287f0639-ccad-4e6f-bc13-fa217a97a345 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:29:14.859283 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98c64ef-e206-4e8a-a7b9-79bd69fc4a93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:29:14.859922 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98c64ef-e206-4e8a-a7b9-79bd69fc4a93 admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:29:14.860208 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98c64ef-e206-4e8a-a7b9-79bd69fc4a93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:29:14.860485 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98c64ef-e206-4e8a-a7b9-79bd69fc4a93 admin admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:29:14.861107 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98c64ef-e206-4e8a-a7b9-79bd69fc4a93 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:29:14.862998 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98c64ef-e206-4e8a-a7b9-79bd69fc4a93 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:29:14.956354 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 996/2001] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 24 09:29:14 2022] POST /identity/v3/projects => generated 375 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:29:14.959191 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e94b39-5e4f-459c-99e7-bc0caead414d None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:29:14.971292 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e94b39-5e4f-459c-99e7-bc0caead414d None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:29:15.022318 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:29:15.037167 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962a5f44-d1d7-48d2-a58a-b5ea1c121952 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:29:15.037792 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962a5f44-d1d7-48d2-a58a-b5ea1c121952 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:29:15.038072 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962a5f44-d1d7-48d2-a58a-b5ea1c121952 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:29:15.038344 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962a5f44-d1d7-48d2-a58a-b5ea1c121952 admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:29:15.038968 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962a5f44-d1d7-48d2-a58a-b5ea1c121952 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:29:15.040905 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962a5f44-d1d7-48d2-a58a-b5ea1c121952 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:29:15.132803 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1006/2002] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 24 09:29:14 2022] POST /identity/v3/users => generated 294 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:29:15.134337 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98c64ef-e206-4e8a-a7b9-79bd69fc4a93 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:29:15.145609 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98c64ef-e206-4e8a-a7b9-79bd69fc4a93 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:29:15.153287 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616bfef8-4fe5-4f93-ac9a-915ac33ca3f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:29:15.154126 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616bfef8-4fe5-4f93-ac9a-915ac33ca3f1 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:29:15.156277 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616bfef8-4fe5-4f93-ac9a-915ac33ca3f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:29:15.156277 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616bfef8-4fe5-4f93-ac9a-915ac33ca3f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:29:15.156277 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616bfef8-4fe5-4f93-ac9a-915ac33ca3f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:29:15.157290 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616bfef8-4fe5-4f93-ac9a-915ac33ca3f1 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:29:15.167883 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 997/2003] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 24 09:29:15 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 09:29:15.172796 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962a5f44-d1d7-48d2-a58a-b5ea1c121952 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:29:15.183537 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962a5f44-d1d7-48d2-a58a-b5ea1c121952 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:29:15.188707 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f34349-2cc4-4df0-87fb-9fcd5fb6d7dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:29:15.189376 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f34349-2cc4-4df0-87fb-9fcd5fb6d7dd admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:29:15.189656 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f34349-2cc4-4df0-87fb-9fcd5fb6d7dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:29:15.189927 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f34349-2cc4-4df0-87fb-9fcd5fb6d7dd admin admin] PATH_INFO: `/v3/projects/568c3e1edbf24df0ae6f271020413b1b/users/822d81cf946f452f8c5769dd819395f1/roles/15f634d4a2d247ddab90c458e5e1c141` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:29:15.206356 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f34349-2cc4-4df0-87fb-9fcd5fb6d7dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=568c3e1edbf24df0ae6f271020413b1b, user_id=822d81cf946f452f8c5769dd819395f1, role_id=15f634d4a2d247ddab90c458e5e1c141)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:29:15.209221 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f34349-2cc4-4df0-87fb-9fcd5fb6d7dd admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:29:15.306304 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1007/2004] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 24 09:29:15 2022] PUT /identity/v3/projects/568c3e1edbf24df0ae6f271020413b1b/users/822d81cf946f452f8c5769dd819395f1/roles/15f634d4a2d247ddab90c458e5e1c141 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:29:15.310411 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616bfef8-4fe5-4f93-ac9a-915ac33ca3f1 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:29:15.320829 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616bfef8-4fe5-4f93-ac9a-915ac33ca3f1 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:29:15.380562 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee596ce5-e445-4816-8924-4fc09e7764a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:29:15.381221 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee596ce5-e445-4816-8924-4fc09e7764a1 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:29:15.381506 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee596ce5-e445-4816-8924-4fc09e7764a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:29:15.381779 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee596ce5-e445-4816-8924-4fc09e7764a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:29:15.382420 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee596ce5-e445-4816-8924-4fc09e7764a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:29:15.384134 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee596ce5-e445-4816-8924-4fc09e7764a1 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:29:15.394413 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 998/2005] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 24 09:29:15 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 09:29:15.398459 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f34349-2cc4-4df0-87fb-9fcd5fb6d7dd admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:29:15.409935 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f34349-2cc4-4df0-87fb-9fcd5fb6d7dd admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:29:15.415084 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50329ad-d7d8-42d5-b0eb-b8da8560a62d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:29:15.415761 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50329ad-d7d8-42d5-b0eb-b8da8560a62d admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:29:15.416422 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50329ad-d7d8-42d5-b0eb-b8da8560a62d admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:29:15.416422 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50329ad-d7d8-42d5-b0eb-b8da8560a62d admin admin] PATH_INFO: `/v3/projects/568c3e1edbf24df0ae6f271020413b1b/users/822d81cf946f452f8c5769dd819395f1/roles/15f634d4a2d247ddab90c458e5e1c141` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:29:15.418773 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50329ad-d7d8-42d5-b0eb-b8da8560a62d admin admin] RBAC: Authorizing `identity:create_grant(project_id=568c3e1edbf24df0ae6f271020413b1b, user_id=822d81cf946f452f8c5769dd819395f1, role_id=15f634d4a2d247ddab90c458e5e1c141)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:29:15.421429 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50329ad-d7d8-42d5-b0eb-b8da8560a62d admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:29:15.432037 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1008/2006] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 24 09:29:15 2022] PUT /identity/v3/projects/568c3e1edbf24df0ae6f271020413b1b/users/822d81cf946f452f8c5769dd819395f1/roles/15f634d4a2d247ddab90c458e5e1c141 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:29:15.443541 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f05cd34-98ae-4b11-b7e1-c817e4e46a11 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:29:15.443865 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f05cd34-98ae-4b11-b7e1-c817e4e46a11 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:29:15.444135 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f05cd34-98ae-4b11-b7e1-c817e4e46a11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:29:15.472752 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-9f05cd34-98ae-4b11-b7e1-c817e4e46a11 None None] MFA Rules not processed for user `822d81cf946f452f8c5769dd819395f1`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:29:15.473991 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-9f05cd34-98ae-4b11-b7e1-c817e4e46a11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:29:15.546219 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:29:15.555350 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 999/2007] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 09:29:15 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:29:15.567311 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50329ad-d7d8-42d5-b0eb-b8da8560a62d admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:29:15.569143 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-d50329ad-d7d8-42d5-b0eb-b8da8560a62d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:29:15.627088 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50329ad-d7d8-42d5-b0eb-b8da8560a62d admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:29:15.630806 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bce7425-cdfb-4b5d-90ce-2b5cd22b99c1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c32b83577c46488906fe409a790b24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:29:15.631410 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bce7425-cdfb-4b5d-90ce-2b5cd22b99c1 service cinder] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:29:15.631723 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bce7425-cdfb-4b5d-90ce-2b5cd22b99c1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:29:15.632001 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bce7425-cdfb-4b5d-90ce-2b5cd22b99c1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:29:15.688591 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bce7425-cdfb-4b5d-90ce-2b5cd22b99c1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:29:15.690159 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bce7425-cdfb-4b5d-90ce-2b5cd22b99c1 service cinder] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:29:15.700680 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1009/2008] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:29:15 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:29:17.591216 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69334a5e-9920-47d2-812f-404bf5ef64f7 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:29:17.591760 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69334a5e-9920-47d2-812f-404bf5ef64f7 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:29:17.591760 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69334a5e-9920-47d2-812f-404bf5ef64f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:29:17.611289 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-69334a5e-9920-47d2-812f-404bf5ef64f7 None None] MFA Rules not processed for user `b3cd284afb854b1e987e807127752925`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:29:17.612558 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-69334a5e-9920-47d2-812f-404bf5ef64f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:29:17.668068 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1000/2009] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 09:29:17 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:29:19.043608 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bce7425-cdfb-4b5d-90ce-2b5cd22b99c1 service cinder] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:29:19.045043 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-2bce7425-cdfb-4b5d-90ce-2b5cd22b99c1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:29:19.056055 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bce7425-cdfb-4b5d-90ce-2b5cd22b99c1 service cinder] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:29:19.060402 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3150b556-4b01-4f69-a6b1-c65df2f56f72 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c32b83577c46488906fe409a790b24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:29:19.071010 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3150b556-4b01-4f69-a6b1-c65df2f56f72 service cinder] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:29:19.071010 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3150b556-4b01-4f69-a6b1-c65df2f56f72 service cinder] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:29:19.071010 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3150b556-4b01-4f69-a6b1-c65df2f56f72 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:29:19.121357 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:29:19.124754 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3150b556-4b01-4f69-a6b1-c65df2f56f72 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:29:19.126188 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3150b556-4b01-4f69-a6b1-c65df2f56f72 service cinder] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:29:19.139173 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1010/2010] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:29:19 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:29:20.828238 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69334a5e-9920-47d2-812f-404bf5ef64f7 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:29:20.838313 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69334a5e-9920-47d2-812f-404bf5ef64f7 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:29:20.841986 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e305340-c456-4e4d-b6e9-8f4a831275c1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c32b83577c46488906fe409a790b24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:29:20.842600 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e305340-c456-4e4d-b6e9-8f4a831275c1 service cinder] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:29:20.842885 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e305340-c456-4e4d-b6e9-8f4a831275c1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:29:20.843166 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e305340-c456-4e4d-b6e9-8f4a831275c1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:29:20.852676 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e305340-c456-4e4d-b6e9-8f4a831275c1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:29:20.854110 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e305340-c456-4e4d-b6e9-8f4a831275c1 service cinder] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:29:20.864757 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1001/2011] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:29:20 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:29:22.078137 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c410cd-4210-4288-9a68-67761e861e04 None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:29:22.079105 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c410cd-4210-4288-9a68-67761e861e04 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:29:22.079393 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c410cd-4210-4288-9a68-67761e861e04 None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:29:22.080834 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1011/2012] 10.222.0.16 () {54 vars in 921 bytes} [Wed Aug 24 09:29:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:29:22.094297 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7107eff-642f-43f2-bd1b-4b103f131842 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:29:22.094824 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7107eff-642f-43f2-bd1b-4b103f131842 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:29:22.094824 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7107eff-642f-43f2-bd1b-4b103f131842 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:29:22.115133 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-e7107eff-642f-43f2-bd1b-4b103f131842 None None] MFA Rules not processed for user `80cada22039d4f11935412ee578acd05`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:29:22.116391 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-e7107eff-642f-43f2-bd1b-4b103f131842 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:29:22.191428 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1002/2013] 10.222.0.16 () {58 vars in 1018 bytes} [Wed Aug 24 09:29:22 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:29:22.215218 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c410cd-4210-4288-9a68-67761e861e04 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:29:22.217099 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-f2c410cd-4210-4288-9a68-67761e861e04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:29:22.236426 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c410cd-4210-4288-9a68-67761e861e04 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:29:22.242315 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e346d0-97fb-4172-9dbb-35757401fe0f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80cada22039d4f11935412ee578acd05', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:29:22.243124 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e346d0-97fb-4172-9dbb-35757401fe0f service placement] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:29:22.243409 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e346d0-97fb-4172-9dbb-35757401fe0f service placement] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:29:22.243725 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e346d0-97fb-4172-9dbb-35757401fe0f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:29:22.253087 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e346d0-97fb-4172-9dbb-35757401fe0f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:29:22.254894 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e346d0-97fb-4172-9dbb-35757401fe0f service placement] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:29:22.267625 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1012/2014] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:29:22 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:29:23.239402 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7107eff-642f-43f2-bd1b-4b103f131842 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:29:23.240907 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-e7107eff-642f-43f2-bd1b-4b103f131842 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:29:23.308753 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7107eff-642f-43f2-bd1b-4b103f131842 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:29:23.313579 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6552dcd4-4ab5-4fe2-8e33-8748e0892bd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:29:23.314296 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6552dcd4-4ab5-4fe2-8e33-8748e0892bd8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:29:23.314580 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6552dcd4-4ab5-4fe2-8e33-8748e0892bd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:29:23.314857 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6552dcd4-4ab5-4fe2-8e33-8748e0892bd8 admin admin] PATH_INFO: `/v3/users/a01fa696ca6a42a1b1432a4a6f162b1d` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:29:23.316027 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6552dcd4-4ab5-4fe2-8e33-8748e0892bd8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a01fa696ca6a42a1b1432a4a6f162b1d)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:29:23.321308 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6552dcd4-4ab5-4fe2-8e33-8748e0892bd8 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:29:23.405105 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:29:23.534386 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-6552dcd4-4ab5-4fe2-8e33-8748e0892bd8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a01fa696ca6a42a1b1432a4a6f162b1d'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:29:23.535282 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-6552dcd4-4ab5-4fe2-8e33-8748e0892bd8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a01fa696ca6a42a1b1432a4a6f162b1d'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:29:23.555361 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-6552dcd4-4ab5-4fe2-8e33-8748e0892bd8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a01fa696ca6a42a1b1432a4a6f162b1d'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:29:23.562971 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-6552dcd4-4ab5-4fe2-8e33-8748e0892bd8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a01fa696ca6a42a1b1432a4a6f162b1d'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:29:23.621134 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-6552dcd4-4ab5-4fe2-8e33-8748e0892bd8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a01fa696ca6a42a1b1432a4a6f162b1d'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:29:23.627460 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1003/2015] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 24 09:29:23 2022] DELETE /identity/v3/users/a01fa696ca6a42a1b1432a4a6f162b1d => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:29:23.945631 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e346d0-97fb-4172-9dbb-35757401fe0f service placement] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:29:23.968968 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e346d0-97fb-4172-9dbb-35757401fe0f service placement] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:29:24.055757 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0901a3a6-19ba-4416-a5fe-67d1b991ae1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:29:24.061010 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0901a3a6-19ba-4416-a5fe-67d1b991ae1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:29:24.061010 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0901a3a6-19ba-4416-a5fe-67d1b991ae1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:29:24.061010 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0901a3a6-19ba-4416-a5fe-67d1b991ae1e admin admin] PATH_INFO: `/v3/projects/7656789d530947a9b05a97e7b0993ff4` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:29:24.061010 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0901a3a6-19ba-4416-a5fe-67d1b991ae1e admin admin] RBAC: Authorizing `identity:delete_project(project_id=7656789d530947a9b05a97e7b0993ff4)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:29:24.061010 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0901a3a6-19ba-4416-a5fe-67d1b991ae1e admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:29:24.187302 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-0901a3a6-19ba-4416-a5fe-67d1b991ae1e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7656789d530947a9b05a97e7b0993ff4', 'request_id': 'req-0901a3a6-19ba-4416-a5fe-67d1b991ae1e'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:29:24.193569 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-0901a3a6-19ba-4416-a5fe-67d1b991ae1e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7656789d530947a9b05a97e7b0993ff4', 'request_id': 'req-0901a3a6-19ba-4416-a5fe-67d1b991ae1e'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:29:24.199727 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-0901a3a6-19ba-4416-a5fe-67d1b991ae1e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7656789d530947a9b05a97e7b0993ff4', 'request_id': 'req-0901a3a6-19ba-4416-a5fe-67d1b991ae1e'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:29:24.278277 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-0901a3a6-19ba-4416-a5fe-67d1b991ae1e admin admin] The token cache is being invalidate because project 7656789d530947a9b05a97e7b0993ff4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81602) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:29:24.278597 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-0901a3a6-19ba-4416-a5fe-67d1b991ae1e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:29:24.279234 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-0901a3a6-19ba-4416-a5fe-67d1b991ae1e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:29:24.281197 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1013/2016] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 24 09:29:23 2022] DELETE /identity/v3/projects/7656789d530947a9b05a97e7b0993ff4 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:29:24.285186 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6552dcd4-4ab5-4fe2-8e33-8748e0892bd8 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:29:24.286679 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-6552dcd4-4ab5-4fe2-8e33-8748e0892bd8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:29:24.300785 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6552dcd4-4ab5-4fe2-8e33-8748e0892bd8 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:29:24.364244 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af28a333-1d7a-40b1-a8b8-23dde27175d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:29:24.365656 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af28a333-1d7a-40b1-a8b8-23dde27175d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:29:24.365656 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af28a333-1d7a-40b1-a8b8-23dde27175d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:29:24.365813 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af28a333-1d7a-40b1-a8b8-23dde27175d0 admin admin] PATH_INFO: `/v3/users/3390875ed30343a393bc15700b5ace57` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:29:24.366924 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af28a333-1d7a-40b1-a8b8-23dde27175d0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3390875ed30343a393bc15700b5ace57)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:29:24.368875 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af28a333-1d7a-40b1-a8b8-23dde27175d0 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:29:24.588104 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-af28a333-1d7a-40b1-a8b8-23dde27175d0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3390875ed30343a393bc15700b5ace57'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:29:24.588758 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-af28a333-1d7a-40b1-a8b8-23dde27175d0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3390875ed30343a393bc15700b5ace57'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:29:24.600442 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-af28a333-1d7a-40b1-a8b8-23dde27175d0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3390875ed30343a393bc15700b5ace57'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:29:24.607753 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-af28a333-1d7a-40b1-a8b8-23dde27175d0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3390875ed30343a393bc15700b5ace57'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:29:24.688443 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-af28a333-1d7a-40b1-a8b8-23dde27175d0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3390875ed30343a393bc15700b5ace57'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:29:24.694621 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1004/2017] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 24 09:29:24 2022] DELETE /identity/v3/users/3390875ed30343a393bc15700b5ace57 => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:29:24.930328 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0901a3a6-19ba-4416-a5fe-67d1b991ae1e admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:29:24.932050 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-0901a3a6-19ba-4416-a5fe-67d1b991ae1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:29:24.947439 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0901a3a6-19ba-4416-a5fe-67d1b991ae1e admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:29:25.006261 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0a5d5c-0f5c-4f8f-a7d3-7e600309771d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:29:25.006902 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0a5d5c-0f5c-4f8f-a7d3-7e600309771d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:29:25.007186 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0a5d5c-0f5c-4f8f-a7d3-7e600309771d admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:29:25.007484 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0a5d5c-0f5c-4f8f-a7d3-7e600309771d admin admin] PATH_INFO: `/v3/projects/bd35a900b44b4cb28f1562e1e275726c` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:29:25.008630 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0a5d5c-0f5c-4f8f-a7d3-7e600309771d admin admin] RBAC: Authorizing `identity:delete_project(project_id=bd35a900b44b4cb28f1562e1e275726c)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:29:25.010555 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0a5d5c-0f5c-4f8f-a7d3-7e600309771d admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:29:25.022670 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:29:25.124474 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-8f0a5d5c-0f5c-4f8f-a7d3-7e600309771d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bd35a900b44b4cb28f1562e1e275726c', 'request_id': 'req-8f0a5d5c-0f5c-4f8f-a7d3-7e600309771d'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:29:25.131985 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-8f0a5d5c-0f5c-4f8f-a7d3-7e600309771d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bd35a900b44b4cb28f1562e1e275726c', 'request_id': 'req-8f0a5d5c-0f5c-4f8f-a7d3-7e600309771d'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:29:25.138567 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-8f0a5d5c-0f5c-4f8f-a7d3-7e600309771d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bd35a900b44b4cb28f1562e1e275726c', 'request_id': 'req-8f0a5d5c-0f5c-4f8f-a7d3-7e600309771d'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:29:25.220945 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-8f0a5d5c-0f5c-4f8f-a7d3-7e600309771d admin admin] The token cache is being invalidate because project bd35a900b44b4cb28f1562e1e275726c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81602) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:29:25.221205 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-8f0a5d5c-0f5c-4f8f-a7d3-7e600309771d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:29:25.221845 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-8f0a5d5c-0f5c-4f8f-a7d3-7e600309771d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:29:25.223371 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1014/2018] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 24 09:29:24 2022] DELETE /identity/v3/projects/bd35a900b44b4cb28f1562e1e275726c => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:29:25.228478 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64faca5e-b2a2-4b5e-a65f-4c35df46f034 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:29:25.229803 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64faca5e-b2a2-4b5e-a65f-4c35df46f034 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:29:25.229909 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64faca5e-b2a2-4b5e-a65f-4c35df46f034 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:29:25.250126 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-64faca5e-b2a2-4b5e-a65f-4c35df46f034 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:29:25.251349 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-64faca5e-b2a2-4b5e-a65f-4c35df46f034 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:29:25.269781 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:DELETE=16,keystone:SELECT=82,keystone:INSERT=2 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:29:25.321550 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1005/2019] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 09:29:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:29:25.329578 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a7b3b1-3bb1-4197-bd49-f680fa76919c None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:29:25.329870 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a7b3b1-3bb1-4197-bd49-f680fa76919c None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:29:25.330503 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a7b3b1-3bb1-4197-bd49-f680fa76919c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:29:25.351240 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-00a7b3b1-3bb1-4197-bd49-f680fa76919c None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:29:25.352471 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-00a7b3b1-3bb1-4197-bd49-f680fa76919c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:29:25.360880 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1015/2020] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 09:29:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:29:25.365174 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64faca5e-b2a2-4b5e-a65f-4c35df46f034 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:29:25.375397 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64faca5e-b2a2-4b5e-a65f-4c35df46f034 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:29:25.380040 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2056ddc3-031a-407f-a909-cbfb72e4f7eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:29:25.380666 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2056ddc3-031a-407f-a909-cbfb72e4f7eb admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:29:25.380943 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2056ddc3-031a-407f-a909-cbfb72e4f7eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:29:25.381214 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2056ddc3-031a-407f-a909-cbfb72e4f7eb admin admin] PATH_INFO: `/v3/domains` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:29:25.381856 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2056ddc3-031a-407f-a909-cbfb72e4f7eb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:29:25.382156 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2056ddc3-031a-407f-a909-cbfb72e4f7eb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:29:25.383766 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2056ddc3-031a-407f-a909-cbfb72e4f7eb admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:29:25.395795 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1006/2021] 10.222.0.16 () {58 vars in 1153 bytes} [Wed Aug 24 09:29:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:29:25.400322 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a7b3b1-3bb1-4197-bd49-f680fa76919c None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:29:25.410569 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a7b3b1-3bb1-4197-bd49-f680fa76919c None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:29:25.415513 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735fc7b7-d71b-48c2-a901-4fa11cb96c91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:29:25.416197 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735fc7b7-d71b-48c2-a901-4fa11cb96c91 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:29:25.416478 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735fc7b7-d71b-48c2-a901-4fa11cb96c91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:29:25.416758 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735fc7b7-d71b-48c2-a901-4fa11cb96c91 admin admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:29:25.417401 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735fc7b7-d71b-48c2-a901-4fa11cb96c91 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:29:25.419414 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735fc7b7-d71b-48c2-a901-4fa11cb96c91 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:29:25.515646 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1016/2022] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 24 09:29:25 2022] POST /identity/v3/projects => generated 375 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:29:25.519372 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2056ddc3-031a-407f-a909-cbfb72e4f7eb admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:29:25.532543 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2056ddc3-031a-407f-a909-cbfb72e4f7eb admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:29:25.615289 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c669c92-9cc6-4552-9673-794b31f4ad35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:29:25.616020 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c669c92-9cc6-4552-9673-794b31f4ad35 admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:29:25.616299 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c669c92-9cc6-4552-9673-794b31f4ad35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:29:25.616598 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c669c92-9cc6-4552-9673-794b31f4ad35 admin admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:29:25.617218 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c669c92-9cc6-4552-9673-794b31f4ad35 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:29:25.619121 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c669c92-9cc6-4552-9673-794b31f4ad35 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:29:25.694655 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1007/2023] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 24 09:29:25 2022] POST /identity/v3/users => generated 288 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 09:29:25.698663 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735fc7b7-d71b-48c2-a901-4fa11cb96c91 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:29:25.709237 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735fc7b7-d71b-48c2-a901-4fa11cb96c91 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:29:25.714130 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2ae268-7f9c-4e9b-82f5-65ba440776c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:29:25.715204 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2ae268-7f9c-4e9b-82f5-65ba440776c8 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:29:25.715535 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2ae268-7f9c-4e9b-82f5-65ba440776c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:29:25.715819 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2ae268-7f9c-4e9b-82f5-65ba440776c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:29:25.716456 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2ae268-7f9c-4e9b-82f5-65ba440776c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:29:25.718034 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2ae268-7f9c-4e9b-82f5-65ba440776c8 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:29:25.728136 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1017/2024] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 24 09:29:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:29:25.732607 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c669c92-9cc6-4552-9673-794b31f4ad35 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:29:25.742825 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c669c92-9cc6-4552-9673-794b31f4ad35 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:29:25.747463 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1047c7a-698c-4e5c-bbe4-50d38cffcea1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:29:25.748092 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1047c7a-698c-4e5c-bbe4-50d38cffcea1 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:29:25.748400 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1047c7a-698c-4e5c-bbe4-50d38cffcea1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:29:25.748675 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1047c7a-698c-4e5c-bbe4-50d38cffcea1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:29:25.749315 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1047c7a-698c-4e5c-bbe4-50d38cffcea1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:29:25.750898 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1047c7a-698c-4e5c-bbe4-50d38cffcea1 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:29:25.761358 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1008/2025] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 24 09:29:25 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:29:25.765199 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2ae268-7f9c-4e9b-82f5-65ba440776c8 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:29:25.774991 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2ae268-7f9c-4e9b-82f5-65ba440776c8 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:29:25.780703 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74445249-5bc7-42ab-a11f-70e06e3d956e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:29:25.781341 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74445249-5bc7-42ab-a11f-70e06e3d956e admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:29:25.781616 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74445249-5bc7-42ab-a11f-70e06e3d956e admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:29:25.781883 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74445249-5bc7-42ab-a11f-70e06e3d956e admin admin] PATH_INFO: `/v3/projects/71b1f6e3a00046bca89ba0e91cdf6c5f/users/0b87ff85d31340ddb0dead3d856a40d9/roles/136cdf78e00a426997dd4e37faff489b` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:29:25.796525 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74445249-5bc7-42ab-a11f-70e06e3d956e admin admin] RBAC: Authorizing `identity:create_grant(project_id=71b1f6e3a00046bca89ba0e91cdf6c5f, user_id=0b87ff85d31340ddb0dead3d856a40d9, role_id=136cdf78e00a426997dd4e37faff489b)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:29:25.799133 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74445249-5bc7-42ab-a11f-70e06e3d956e admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:29:25.906469 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1018/2026] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 24 09:29:25 2022] PUT /identity/v3/projects/71b1f6e3a00046bca89ba0e91cdf6c5f/users/0b87ff85d31340ddb0dead3d856a40d9/roles/136cdf78e00a426997dd4e37faff489b => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:29:25.916661 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a3fe1c-06f3-4877-9e8c-21f076e8d6da None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:29:25.916948 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a3fe1c-06f3-4877-9e8c-21f076e8d6da None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:29:25.917228 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a3fe1c-06f3-4877-9e8c-21f076e8d6da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:29:25.944386 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-27a3fe1c-06f3-4877-9e8c-21f076e8d6da None None] MFA Rules not processed for user `0b87ff85d31340ddb0dead3d856a40d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:29:25.945587 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-27a3fe1c-06f3-4877-9e8c-21f076e8d6da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:29:26.009021 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1009/2027] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 09:29:25 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 09:29:26.020321 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74445249-5bc7-42ab-a11f-70e06e3d956e admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:29:26.022080 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-74445249-5bc7-42ab-a11f-70e06e3d956e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:29:26.099293 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74445249-5bc7-42ab-a11f-70e06e3d956e admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:29:26.103001 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2212cda1-8141-4dca-9c73-01300425a354 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea2666d430a646d386104addf1f42888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:29:26.103629 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2212cda1-8141-4dca-9c73-01300425a354 service glance] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:29:26.103911 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2212cda1-8141-4dca-9c73-01300425a354 service glance] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:29:26.104182 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2212cda1-8141-4dca-9c73-01300425a354 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:29:26.112883 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2212cda1-8141-4dca-9c73-01300425a354 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:29:26.114302 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2212cda1-8141-4dca-9c73-01300425a354 service glance] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:29:26.123720 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1019/2028] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:29:26 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:29:29.810969 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a3fe1c-06f3-4877-9e8c-21f076e8d6da None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:29:29.813616 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-27a3fe1c-06f3-4877-9e8c-21f076e8d6da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:29:29.842694 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:29:29.903436 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a3fe1c-06f3-4877-9e8c-21f076e8d6da None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:29:29.910099 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276bdc17-6f90-4110-9bd3-900edac5eb80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:29:29.910948 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276bdc17-6f90-4110-9bd3-900edac5eb80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:29:29.912093 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276bdc17-6f90-4110-9bd3-900edac5eb80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:29:29.912813 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276bdc17-6f90-4110-9bd3-900edac5eb80 admin admin] PATH_INFO: `/v3/users/1cd6bfe7f1fa4487afe63292fb1f02f8` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:29:29.915317 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276bdc17-6f90-4110-9bd3-900edac5eb80 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1cd6bfe7f1fa4487afe63292fb1f02f8)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:29:29.918677 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276bdc17-6f90-4110-9bd3-900edac5eb80 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:29:30.091377 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-276bdc17-6f90-4110-9bd3-900edac5eb80 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1cd6bfe7f1fa4487afe63292fb1f02f8'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:29:30.092083 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-276bdc17-6f90-4110-9bd3-900edac5eb80 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1cd6bfe7f1fa4487afe63292fb1f02f8'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:29:30.109057 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-276bdc17-6f90-4110-9bd3-900edac5eb80 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1cd6bfe7f1fa4487afe63292fb1f02f8'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:29:30.117497 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-276bdc17-6f90-4110-9bd3-900edac5eb80 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1cd6bfe7f1fa4487afe63292fb1f02f8'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:29:30.209213 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-276bdc17-6f90-4110-9bd3-900edac5eb80 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1cd6bfe7f1fa4487afe63292fb1f02f8'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:29:30.217677 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1010/2029] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 24 09:29:29 2022] DELETE /identity/v3/users/1cd6bfe7f1fa4487afe63292fb1f02f8 => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:29:30.222112 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3f5d61-60c0-46d2-9057-19c7de7ae169 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:29:30.227613 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3f5d61-60c0-46d2-9057-19c7de7ae169 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:29:30.227613 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3f5d61-60c0-46d2-9057-19c7de7ae169 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:29:30.244002 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-8e3f5d61-60c0-46d2-9057-19c7de7ae169 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:29:30.245391 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-8e3f5d61-60c0-46d2-9057-19c7de7ae169 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:29:30.258847 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:29:30.315001 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1020/2030] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 09:29:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:29:30.327439 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276bdc17-6f90-4110-9bd3-900edac5eb80 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:29:30.329466 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-276bdc17-6f90-4110-9bd3-900edac5eb80 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:29:30.400099 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276bdc17-6f90-4110-9bd3-900edac5eb80 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:29:30.406290 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedda3e7-c8ff-4dd9-a434-3b2d2447741d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e5260e798404e8bb12b25ca8ea97052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:29:30.406973 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedda3e7-c8ff-4dd9-a434-3b2d2447741d service neutron] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:29:30.407261 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedda3e7-c8ff-4dd9-a434-3b2d2447741d service neutron] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:29:30.407600 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedda3e7-c8ff-4dd9-a434-3b2d2447741d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:29:30.417565 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedda3e7-c8ff-4dd9-a434-3b2d2447741d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:29:30.419202 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedda3e7-c8ff-4dd9-a434-3b2d2447741d service neutron] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:29:30.431116 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1011/2031] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:29:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 09:29:30.874342 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3f5d61-60c0-46d2-9057-19c7de7ae169 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:29:30.876121 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-8e3f5d61-60c0-46d2-9057-19c7de7ae169 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:29:30.888693 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3f5d61-60c0-46d2-9057-19c7de7ae169 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:29:30.894529 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364e3e76-02c9-4800-bf2b-abd044a1a3c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:29:30.895316 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364e3e76-02c9-4800-bf2b-abd044a1a3c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:29:30.895704 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364e3e76-02c9-4800-bf2b-abd044a1a3c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:29:30.896061 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364e3e76-02c9-4800-bf2b-abd044a1a3c3 admin admin] PATH_INFO: `/v3/projects/6107a70b787c455d8fa96b065ba42cb7` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:29:30.897321 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364e3e76-02c9-4800-bf2b-abd044a1a3c3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6107a70b787c455d8fa96b065ba42cb7)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:29:30.899441 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364e3e76-02c9-4800-bf2b-abd044a1a3c3 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:29:31.026307 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-364e3e76-02c9-4800-bf2b-abd044a1a3c3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6107a70b787c455d8fa96b065ba42cb7', 'request_id': 'req-364e3e76-02c9-4800-bf2b-abd044a1a3c3'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:29:31.033447 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-364e3e76-02c9-4800-bf2b-abd044a1a3c3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6107a70b787c455d8fa96b065ba42cb7', 'request_id': 'req-364e3e76-02c9-4800-bf2b-abd044a1a3c3'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:29:31.039322 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-364e3e76-02c9-4800-bf2b-abd044a1a3c3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6107a70b787c455d8fa96b065ba42cb7', 'request_id': 'req-364e3e76-02c9-4800-bf2b-abd044a1a3c3'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:29:31.133821 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-364e3e76-02c9-4800-bf2b-abd044a1a3c3 admin admin] The token cache is being invalidate because project 6107a70b787c455d8fa96b065ba42cb7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81602) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:29:31.136623 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-364e3e76-02c9-4800-bf2b-abd044a1a3c3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:29:31.136623 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-364e3e76-02c9-4800-bf2b-abd044a1a3c3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:29:31.136623 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1021/2032] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 24 09:29:30 2022] DELETE /identity/v3/projects/6107a70b787c455d8fa96b065ba42cb7 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:29:31.140507 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedda3e7-c8ff-4dd9-a434-3b2d2447741d service neutron] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:29:31.157979 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedda3e7-c8ff-4dd9-a434-3b2d2447741d service neutron] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:29:31.193764 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:29:31.216868 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22d1dcc-a018-4542-adbc-ab15f01671fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:29:31.216868 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22d1dcc-a018-4542-adbc-ab15f01671fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:29:31.216868 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22d1dcc-a018-4542-adbc-ab15f01671fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:29:31.216868 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22d1dcc-a018-4542-adbc-ab15f01671fb admin admin] PATH_INFO: `/v3/users/cfccf1385b684eb9b2d40c1cb3c2e54c` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:29:31.217292 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22d1dcc-a018-4542-adbc-ab15f01671fb admin admin] RBAC: Authorizing `identity:delete_user(user_id=cfccf1385b684eb9b2d40c1cb3c2e54c)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:29:31.217787 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22d1dcc-a018-4542-adbc-ab15f01671fb admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:29:31.388815 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-e22d1dcc-a018-4542-adbc-ab15f01671fb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cfccf1385b684eb9b2d40c1cb3c2e54c'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:29:31.389473 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-e22d1dcc-a018-4542-adbc-ab15f01671fb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cfccf1385b684eb9b2d40c1cb3c2e54c'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:29:31.401291 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-e22d1dcc-a018-4542-adbc-ab15f01671fb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cfccf1385b684eb9b2d40c1cb3c2e54c'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:29:31.408748 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-e22d1dcc-a018-4542-adbc-ab15f01671fb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cfccf1385b684eb9b2d40c1cb3c2e54c'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:29:31.506105 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-e22d1dcc-a018-4542-adbc-ab15f01671fb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cfccf1385b684eb9b2d40c1cb3c2e54c'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:29:31.513044 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1012/2033] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 24 09:29:31 2022] DELETE /identity/v3/users/cfccf1385b684eb9b2d40c1cb3c2e54c => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:29:31.936889 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364e3e76-02c9-4800-bf2b-abd044a1a3c3 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:29:31.938379 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-364e3e76-02c9-4800-bf2b-abd044a1a3c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:29:31.953863 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364e3e76-02c9-4800-bf2b-abd044a1a3c3 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:29:32.020514 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab190d4-e84f-4dd2-b1e6-610024363739 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:29:32.021186 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab190d4-e84f-4dd2-b1e6-610024363739 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:29:32.021463 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab190d4-e84f-4dd2-b1e6-610024363739 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:29:32.021739 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab190d4-e84f-4dd2-b1e6-610024363739 admin admin] PATH_INFO: `/v3/projects/b69eb735ad95443095fe7f0ffd6b9c06` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:29:32.022892 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab190d4-e84f-4dd2-b1e6-610024363739 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b69eb735ad95443095fe7f0ffd6b9c06)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:29:32.024840 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab190d4-e84f-4dd2-b1e6-610024363739 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:29:32.151736 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:29:32.154810 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-bab190d4-e84f-4dd2-b1e6-610024363739 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b69eb735ad95443095fe7f0ffd6b9c06', 'request_id': 'req-bab190d4-e84f-4dd2-b1e6-610024363739'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:29:32.160957 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-bab190d4-e84f-4dd2-b1e6-610024363739 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b69eb735ad95443095fe7f0ffd6b9c06', 'request_id': 'req-bab190d4-e84f-4dd2-b1e6-610024363739'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:29:32.166332 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-bab190d4-e84f-4dd2-b1e6-610024363739 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b69eb735ad95443095fe7f0ffd6b9c06', 'request_id': 'req-bab190d4-e84f-4dd2-b1e6-610024363739'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:29:32.260613 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-bab190d4-e84f-4dd2-b1e6-610024363739 admin admin] The token cache is being invalidate because project b69eb735ad95443095fe7f0ffd6b9c06 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81602) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:29:32.260923 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-bab190d4-e84f-4dd2-b1e6-610024363739 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:29:32.261814 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-bab190d4-e84f-4dd2-b1e6-610024363739 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:29:32.263379 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1022/2034] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 24 09:29:31 2022] DELETE /identity/v3/projects/b69eb735ad95443095fe7f0ffd6b9c06 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:29:32.268041 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22d1dcc-a018-4542-adbc-ab15f01671fb admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:29:32.270097 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-e22d1dcc-a018-4542-adbc-ab15f01671fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:29:32.282926 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22d1dcc-a018-4542-adbc-ab15f01671fb admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:29:32.348595 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86e850d-08ab-4f2c-a6cc-5199ef37e5bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:29:32.349225 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86e850d-08ab-4f2c-a6cc-5199ef37e5bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:29:32.349514 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86e850d-08ab-4f2c-a6cc-5199ef37e5bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:29:32.349790 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86e850d-08ab-4f2c-a6cc-5199ef37e5bc admin admin] PATH_INFO: `/v3/users/822d81cf946f452f8c5769dd819395f1` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:29:32.350921 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86e850d-08ab-4f2c-a6cc-5199ef37e5bc admin admin] RBAC: Authorizing `identity:delete_user(user_id=822d81cf946f452f8c5769dd819395f1)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:29:32.352891 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86e850d-08ab-4f2c-a6cc-5199ef37e5bc admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:29:32.532248 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-b86e850d-08ab-4f2c-a6cc-5199ef37e5bc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '822d81cf946f452f8c5769dd819395f1'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:29:32.532908 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-b86e850d-08ab-4f2c-a6cc-5199ef37e5bc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '822d81cf946f452f8c5769dd819395f1'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:29:32.543970 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=81,keystone:DELETE=18,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:29:32.551120 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-b86e850d-08ab-4f2c-a6cc-5199ef37e5bc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '822d81cf946f452f8c5769dd819395f1'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:29:32.552930 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-b86e850d-08ab-4f2c-a6cc-5199ef37e5bc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '822d81cf946f452f8c5769dd819395f1'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:29:32.609026 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-b86e850d-08ab-4f2c-a6cc-5199ef37e5bc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '822d81cf946f452f8c5769dd819395f1'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:29:32.615409 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1013/2035] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 24 09:29:32 2022] DELETE /identity/v3/users/822d81cf946f452f8c5769dd819395f1 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:29:33.010663 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab190d4-e84f-4dd2-b1e6-610024363739 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:29:33.013275 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-bab190d4-e84f-4dd2-b1e6-610024363739 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:29:33.031874 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab190d4-e84f-4dd2-b1e6-610024363739 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:29:33.090631 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c6f56a-1837-4770-bbfa-479364e30052 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:29:33.091269 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c6f56a-1837-4770-bbfa-479364e30052 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:29:33.091556 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c6f56a-1837-4770-bbfa-479364e30052 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:29:33.091860 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c6f56a-1837-4770-bbfa-479364e30052 admin admin] PATH_INFO: `/v3/projects/568c3e1edbf24df0ae6f271020413b1b` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:29:33.093004 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c6f56a-1837-4770-bbfa-479364e30052 admin admin] RBAC: Authorizing `identity:delete_project(project_id=568c3e1edbf24df0ae6f271020413b1b)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:29:33.094910 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c6f56a-1837-4770-bbfa-479364e30052 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:29:33.231932 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-07c6f56a-1837-4770-bbfa-479364e30052 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '568c3e1edbf24df0ae6f271020413b1b', 'request_id': 'req-07c6f56a-1837-4770-bbfa-479364e30052'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:29:33.239943 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-07c6f56a-1837-4770-bbfa-479364e30052 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '568c3e1edbf24df0ae6f271020413b1b', 'request_id': 'req-07c6f56a-1837-4770-bbfa-479364e30052'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:29:33.246660 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-07c6f56a-1837-4770-bbfa-479364e30052 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '568c3e1edbf24df0ae6f271020413b1b', 'request_id': 'req-07c6f56a-1837-4770-bbfa-479364e30052'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:29:33.338172 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-07c6f56a-1837-4770-bbfa-479364e30052 admin admin] The token cache is being invalidate because project 568c3e1edbf24df0ae6f271020413b1b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81602) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:29:33.338584 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-07c6f56a-1837-4770-bbfa-479364e30052 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:29:33.339343 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-07c6f56a-1837-4770-bbfa-479364e30052 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:29:33.341184 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1023/2036] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 24 09:29:33 2022] DELETE /identity/v3/projects/568c3e1edbf24df0ae6f271020413b1b => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:29:33.346038 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d86904-fbf5-464b-91e3-b47d31dc3f43 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:29:33.348037 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d86904-fbf5-464b-91e3-b47d31dc3f43 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:29:33.348037 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d86904-fbf5-464b-91e3-b47d31dc3f43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:29:33.365674 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-37d86904-fbf5-464b-91e3-b47d31dc3f43 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:29:33.371421 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-37d86904-fbf5-464b-91e3-b47d31dc3f43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:29:33.434298 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1014/2037] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 09:29:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 09:29:33.442045 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94716947-79c2-4e31-94ac-3a80330d8320 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:29:33.442434 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94716947-79c2-4e31-94ac-3a80330d8320 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:29:33.442602 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94716947-79c2-4e31-94ac-3a80330d8320 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:29:33.463706 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-94716947-79c2-4e31-94ac-3a80330d8320 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:29:33.465002 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-94716947-79c2-4e31-94ac-3a80330d8320 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:29:33.473268 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1024/2038] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 09:29:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:29:33.477767 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d86904-fbf5-464b-91e3-b47d31dc3f43 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:29:33.488224 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d86904-fbf5-464b-91e3-b47d31dc3f43 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:29:33.492921 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751ffeaf-96a6-4fdf-a710-401ecb7500ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:29:33.493593 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751ffeaf-96a6-4fdf-a710-401ecb7500ca admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:29:33.493875 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751ffeaf-96a6-4fdf-a710-401ecb7500ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:29:33.494145 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751ffeaf-96a6-4fdf-a710-401ecb7500ca admin admin] PATH_INFO: `/v3/domains` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:29:33.494801 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751ffeaf-96a6-4fdf-a710-401ecb7500ca admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:29:33.495103 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751ffeaf-96a6-4fdf-a710-401ecb7500ca admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:29:33.497306 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751ffeaf-96a6-4fdf-a710-401ecb7500ca admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:29:33.508765 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1015/2039] 10.222.0.16 () {58 vars in 1153 bytes} [Wed Aug 24 09:29:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:29:33.513409 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94716947-79c2-4e31-94ac-3a80330d8320 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:29:33.522783 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94716947-79c2-4e31-94ac-3a80330d8320 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:29:33.527502 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063c4a29-fd28-4cd1-becb-9bba4aba2c19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:29:33.528128 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063c4a29-fd28-4cd1-becb-9bba4aba2c19 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:29:33.528403 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063c4a29-fd28-4cd1-becb-9bba4aba2c19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:29:33.528675 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063c4a29-fd28-4cd1-becb-9bba4aba2c19 admin admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:29:33.529329 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063c4a29-fd28-4cd1-becb-9bba4aba2c19 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:29:33.531196 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063c4a29-fd28-4cd1-becb-9bba4aba2c19 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:29:33.634035 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1025/2040] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 24 09:29:33 2022] POST /identity/v3/projects => generated 369 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 09:29:33.638212 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751ffeaf-96a6-4fdf-a710-401ecb7500ca admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:29:33.648685 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751ffeaf-96a6-4fdf-a710-401ecb7500ca admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:29:33.706107 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553f798c-c100-4dd1-9ec5-11b02ee5f7e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:29:33.706712 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553f798c-c100-4dd1-9ec5-11b02ee5f7e5 admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:29:33.706998 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553f798c-c100-4dd1-9ec5-11b02ee5f7e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:29:33.707281 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553f798c-c100-4dd1-9ec5-11b02ee5f7e5 admin admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:29:33.708027 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553f798c-c100-4dd1-9ec5-11b02ee5f7e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:29:33.710026 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553f798c-c100-4dd1-9ec5-11b02ee5f7e5 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:29:33.878872 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1016/2041] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 24 09:29:33 2022] POST /identity/v3/users => generated 285 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:29:33.882768 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063c4a29-fd28-4cd1-becb-9bba4aba2c19 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:29:33.893149 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063c4a29-fd28-4cd1-becb-9bba4aba2c19 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:29:33.897888 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909324b1-23c8-4c6a-b431-4f8712311af9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:29:33.898569 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909324b1-23c8-4c6a-b431-4f8712311af9 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:29:33.898797 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909324b1-23c8-4c6a-b431-4f8712311af9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:29:33.899188 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909324b1-23c8-4c6a-b431-4f8712311af9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:29:33.901334 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909324b1-23c8-4c6a-b431-4f8712311af9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:29:33.901334 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909324b1-23c8-4c6a-b431-4f8712311af9 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:29:33.914236 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1026/2042] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 24 09:29:33 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:29:33.920117 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553f798c-c100-4dd1-9ec5-11b02ee5f7e5 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:29:33.930955 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553f798c-c100-4dd1-9ec5-11b02ee5f7e5 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:29:33.935773 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224ecbcd-abbb-4dcd-973f-c83c70b0c4f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:29:33.936369 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224ecbcd-abbb-4dcd-973f-c83c70b0c4f4 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:29:33.936649 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224ecbcd-abbb-4dcd-973f-c83c70b0c4f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:29:33.936921 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224ecbcd-abbb-4dcd-973f-c83c70b0c4f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:29:33.937557 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224ecbcd-abbb-4dcd-973f-c83c70b0c4f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:29:33.939305 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224ecbcd-abbb-4dcd-973f-c83c70b0c4f4 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:29:33.950312 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1017/2043] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 24 09:29:33 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:29:33.956831 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909324b1-23c8-4c6a-b431-4f8712311af9 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:29:33.967608 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909324b1-23c8-4c6a-b431-4f8712311af9 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:29:33.972301 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e2a75e-06ec-4ed6-b688-cb7707b96ee3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:29:33.973021 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e2a75e-06ec-4ed6-b688-cb7707b96ee3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:29:33.973332 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e2a75e-06ec-4ed6-b688-cb7707b96ee3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:29:33.973727 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e2a75e-06ec-4ed6-b688-cb7707b96ee3 admin admin] PATH_INFO: `/v3/projects/d1d26b18d3d0403797d1fd0d3ccdf85d/users/a9a36d5a7e4b4178807fd9f0fab2f350/roles/136cdf78e00a426997dd4e37faff489b` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:29:33.993988 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e2a75e-06ec-4ed6-b688-cb7707b96ee3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d1d26b18d3d0403797d1fd0d3ccdf85d, user_id=a9a36d5a7e4b4178807fd9f0fab2f350, role_id=136cdf78e00a426997dd4e37faff489b)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:29:33.993988 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e2a75e-06ec-4ed6-b688-cb7707b96ee3 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:29:34.033553 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:DELETE=11,keystone:SELECT=85,keystone:INSERT=4 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:29:34.038351 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1027/2044] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 24 09:29:33 2022] PUT /identity/v3/projects/d1d26b18d3d0403797d1fd0d3ccdf85d/users/a9a36d5a7e4b4178807fd9f0fab2f350/roles/136cdf78e00a426997dd4e37faff489b => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:29:34.049983 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648a1bad-b9a6-47f2-8e51-95c3548fff66 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:29:34.050355 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648a1bad-b9a6-47f2-8e51-95c3548fff66 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:29:34.050741 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648a1bad-b9a6-47f2-8e51-95c3548fff66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:29:34.084228 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-648a1bad-b9a6-47f2-8e51-95c3548fff66 None None] MFA Rules not processed for user `a9a36d5a7e4b4178807fd9f0fab2f350`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:29:34.085503 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-648a1bad-b9a6-47f2-8e51-95c3548fff66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:29:34.121789 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:29:34.158705 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1018/2045] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 09:29:34 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:29:34.176771 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e2a75e-06ec-4ed6-b688-cb7707b96ee3 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:29:34.178773 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-02e2a75e-06ec-4ed6-b688-cb7707b96ee3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:29:34.237533 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e2a75e-06ec-4ed6-b688-cb7707b96ee3 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:29:34.241692 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bcdda6-25c3-4459-bf93-f61c5e5e7559 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea2666d430a646d386104addf1f42888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:29:34.245594 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bcdda6-25c3-4459-bf93-f61c5e5e7559 service glance] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:29:34.245594 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bcdda6-25c3-4459-bf93-f61c5e5e7559 service glance] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:29:34.245594 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bcdda6-25c3-4459-bf93-f61c5e5e7559 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:29:34.252193 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bcdda6-25c3-4459-bf93-f61c5e5e7559 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:29:34.253650 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bcdda6-25c3-4459-bf93-f61c5e5e7559 service glance] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:29:34.263661 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1028/2046] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:29:34 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:29:36.406873 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648a1bad-b9a6-47f2-8e51-95c3548fff66 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:29:36.408244 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-648a1bad-b9a6-47f2-8e51-95c3548fff66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:29:36.502631 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648a1bad-b9a6-47f2-8e51-95c3548fff66 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:29:36.507693 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cf9f31-3668-4a38-a2d7-bc2278809676 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:29:36.508602 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cf9f31-3668-4a38-a2d7-bc2278809676 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:29:36.508889 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cf9f31-3668-4a38-a2d7-bc2278809676 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:29:36.509166 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cf9f31-3668-4a38-a2d7-bc2278809676 admin admin] PATH_INFO: `/v3/users/0525c1879f3a4b44a131daac894680a0` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:29:36.510359 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cf9f31-3668-4a38-a2d7-bc2278809676 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0525c1879f3a4b44a131daac894680a0)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:29:36.512408 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cf9f31-3668-4a38-a2d7-bc2278809676 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:29:36.652159 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-97cf9f31-3668-4a38-a2d7-bc2278809676 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0525c1879f3a4b44a131daac894680a0'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:29:36.652959 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-97cf9f31-3668-4a38-a2d7-bc2278809676 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0525c1879f3a4b44a131daac894680a0'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:29:36.666835 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-97cf9f31-3668-4a38-a2d7-bc2278809676 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0525c1879f3a4b44a131daac894680a0'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:29:36.675028 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-97cf9f31-3668-4a38-a2d7-bc2278809676 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0525c1879f3a4b44a131daac894680a0'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:29:36.757269 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-97cf9f31-3668-4a38-a2d7-bc2278809676 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0525c1879f3a4b44a131daac894680a0'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:29:36.764817 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1019/2047] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 24 09:29:36 2022] DELETE /identity/v3/users/0525c1879f3a4b44a131daac894680a0 => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:29:36.770159 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6975f0-8bcb-4694-8c14-6e9b07209ff0 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:29:36.771162 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6975f0-8bcb-4694-8c14-6e9b07209ff0 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:29:36.772066 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6975f0-8bcb-4694-8c14-6e9b07209ff0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:29:36.797734 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-eb6975f0-8bcb-4694-8c14-6e9b07209ff0 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:29:36.800026 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-eb6975f0-8bcb-4694-8c14-6e9b07209ff0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:29:36.890690 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1029/2048] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 09:29:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:29:36.901246 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cf9f31-3668-4a38-a2d7-bc2278809676 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:29:36.903267 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-97cf9f31-3668-4a38-a2d7-bc2278809676 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:29:36.967032 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cf9f31-3668-4a38-a2d7-bc2278809676 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:29:36.970644 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1ea020-fe38-4d3e-bbdd-097e1ccbe06a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e5260e798404e8bb12b25ca8ea97052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:29:36.971296 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1ea020-fe38-4d3e-bbdd-097e1ccbe06a service neutron] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:29:36.971628 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1ea020-fe38-4d3e-bbdd-097e1ccbe06a service neutron] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:29:36.971908 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1ea020-fe38-4d3e-bbdd-097e1ccbe06a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:29:36.979301 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:29:36.982513 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1ea020-fe38-4d3e-bbdd-097e1ccbe06a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:29:36.984090 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1ea020-fe38-4d3e-bbdd-097e1ccbe06a service neutron] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:29:36.999902 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1020/2049] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:29:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:29:37.456578 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6975f0-8bcb-4694-8c14-6e9b07209ff0 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:29:37.463623 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-eb6975f0-8bcb-4694-8c14-6e9b07209ff0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:29:37.482657 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6975f0-8bcb-4694-8c14-6e9b07209ff0 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:29:37.488786 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecef843-78d1-48ae-a9c4-d0b20bbbf601 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:29:37.489432 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecef843-78d1-48ae-a9c4-d0b20bbbf601 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:29:37.489714 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecef843-78d1-48ae-a9c4-d0b20bbbf601 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:29:37.489990 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecef843-78d1-48ae-a9c4-d0b20bbbf601 admin admin] PATH_INFO: `/v3/projects/83b7398cf2994a328ae9ba896956a6fe` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:29:37.491370 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecef843-78d1-48ae-a9c4-d0b20bbbf601 admin admin] RBAC: Authorizing `identity:delete_project(project_id=83b7398cf2994a328ae9ba896956a6fe)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:29:37.493350 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecef843-78d1-48ae-a9c4-d0b20bbbf601 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:29:37.639792 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-7ecef843-78d1-48ae-a9c4-d0b20bbbf601 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '83b7398cf2994a328ae9ba896956a6fe', 'request_id': 'req-7ecef843-78d1-48ae-a9c4-d0b20bbbf601'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:29:37.654232 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-7ecef843-78d1-48ae-a9c4-d0b20bbbf601 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '83b7398cf2994a328ae9ba896956a6fe', 'request_id': 'req-7ecef843-78d1-48ae-a9c4-d0b20bbbf601'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:29:37.661348 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-7ecef843-78d1-48ae-a9c4-d0b20bbbf601 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '83b7398cf2994a328ae9ba896956a6fe', 'request_id': 'req-7ecef843-78d1-48ae-a9c4-d0b20bbbf601'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:29:37.733849 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-7ecef843-78d1-48ae-a9c4-d0b20bbbf601 admin admin] The token cache is being invalidate because project 83b7398cf2994a328ae9ba896956a6fe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81602) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:29:37.734202 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-7ecef843-78d1-48ae-a9c4-d0b20bbbf601 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:29:37.735456 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-7ecef843-78d1-48ae-a9c4-d0b20bbbf601 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:29:37.736481 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1030/2050] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 24 09:29:37 2022] DELETE /identity/v3/projects/83b7398cf2994a328ae9ba896956a6fe => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:29:37.744260 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d52a6c5-2a09-4749-922e-8809e1c88db4 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:29:37.744806 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d52a6c5-2a09-4749-922e-8809e1c88db4 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:29:37.744923 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d52a6c5-2a09-4749-922e-8809e1c88db4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:29:37.770536 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-0d52a6c5-2a09-4749-922e-8809e1c88db4 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:29:37.773231 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-0d52a6c5-2a09-4749-922e-8809e1c88db4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:29:37.840484 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1021/2051] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 09:29:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:29:37.849138 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1adb766-eca5-4b69-9fda-788ca0e09294 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:29:37.849641 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1adb766-eca5-4b69-9fda-788ca0e09294 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:29:37.849751 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1adb766-eca5-4b69-9fda-788ca0e09294 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:29:37.869876 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-b1adb766-eca5-4b69-9fda-788ca0e09294 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:29:37.871651 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-b1adb766-eca5-4b69-9fda-788ca0e09294 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:29:37.882815 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1031/2052] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 09:29:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:29:37.887727 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d52a6c5-2a09-4749-922e-8809e1c88db4 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:29:37.900653 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d52a6c5-2a09-4749-922e-8809e1c88db4 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:29:37.905608 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b0cfc8-4f25-4f1d-a8fa-542531254ead admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:29:37.906220 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b0cfc8-4f25-4f1d-a8fa-542531254ead admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:29:37.906501 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b0cfc8-4f25-4f1d-a8fa-542531254ead admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:29:37.906763 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b0cfc8-4f25-4f1d-a8fa-542531254ead admin admin] PATH_INFO: `/v3/domains` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:29:37.907412 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b0cfc8-4f25-4f1d-a8fa-542531254ead admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:29:37.907754 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b0cfc8-4f25-4f1d-a8fa-542531254ead admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:29:37.909352 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b0cfc8-4f25-4f1d-a8fa-542531254ead admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:29:37.923420 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1022/2053] 10.222.0.16 () {58 vars in 1153 bytes} [Wed Aug 24 09:29:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:29:37.929658 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1adb766-eca5-4b69-9fda-788ca0e09294 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:29:37.939081 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1adb766-eca5-4b69-9fda-788ca0e09294 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:29:37.944987 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f400f5f7-cc9d-4f37-97cb-bed859c60f3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:29:37.945606 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f400f5f7-cc9d-4f37-97cb-bed859c60f3c admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:29:37.945904 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f400f5f7-cc9d-4f37-97cb-bed859c60f3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:29:37.948305 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f400f5f7-cc9d-4f37-97cb-bed859c60f3c admin admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:29:37.949569 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f400f5f7-cc9d-4f37-97cb-bed859c60f3c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:29:37.951705 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f400f5f7-cc9d-4f37-97cb-bed859c60f3c admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:29:38.044410 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:29:38.055483 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1032/2054] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 24 09:29:37 2022] POST /identity/v3/projects => generated 385 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 09:29:38.060347 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b0cfc8-4f25-4f1d-a8fa-542531254ead admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:29:38.076121 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b0cfc8-4f25-4f1d-a8fa-542531254ead admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:29:38.159519 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381cc1ed-bed5-4493-aa72-514284b61fe0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:29:38.160201 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381cc1ed-bed5-4493-aa72-514284b61fe0 admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:29:38.160485 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381cc1ed-bed5-4493-aa72-514284b61fe0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:29:38.160762 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381cc1ed-bed5-4493-aa72-514284b61fe0 admin admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:29:38.161397 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381cc1ed-bed5-4493-aa72-514284b61fe0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:29:38.164232 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381cc1ed-bed5-4493-aa72-514284b61fe0 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:29:38.255059 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1023/2055] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 24 09:29:38 2022] POST /identity/v3/users => generated 293 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:29:38.259278 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f400f5f7-cc9d-4f37-97cb-bed859c60f3c admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:29:38.269840 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f400f5f7-cc9d-4f37-97cb-bed859c60f3c admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:29:38.274888 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d247e139-0b93-42eb-ad41-a9cdb152fbc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:29:38.275835 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d247e139-0b93-42eb-ad41-a9cdb152fbc6 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:29:38.275835 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d247e139-0b93-42eb-ad41-a9cdb152fbc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:29:38.276124 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d247e139-0b93-42eb-ad41-a9cdb152fbc6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:29:38.276840 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d247e139-0b93-42eb-ad41-a9cdb152fbc6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:29:38.278516 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d247e139-0b93-42eb-ad41-a9cdb152fbc6 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:29:38.291371 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1033/2056] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 24 09:29:38 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:29:38.296140 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381cc1ed-bed5-4493-aa72-514284b61fe0 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:29:38.306474 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381cc1ed-bed5-4493-aa72-514284b61fe0 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:29:38.311880 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f38fc0c-22e3-41fe-bc9f-cdf331ea0ea3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:29:38.312480 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f38fc0c-22e3-41fe-bc9f-cdf331ea0ea3 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:29:38.312783 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f38fc0c-22e3-41fe-bc9f-cdf331ea0ea3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:29:38.313044 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f38fc0c-22e3-41fe-bc9f-cdf331ea0ea3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:29:38.313695 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f38fc0c-22e3-41fe-bc9f-cdf331ea0ea3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:29:38.315289 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f38fc0c-22e3-41fe-bc9f-cdf331ea0ea3 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:29:38.325845 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1024/2057] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 24 09:29:38 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:29:38.330124 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d247e139-0b93-42eb-ad41-a9cdb152fbc6 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:29:38.340746 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d247e139-0b93-42eb-ad41-a9cdb152fbc6 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:29:38.345394 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4aba78-a16a-496d-8a43-72daf6c6f1bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:29:38.346086 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4aba78-a16a-496d-8a43-72daf6c6f1bc admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:29:38.346365 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4aba78-a16a-496d-8a43-72daf6c6f1bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:29:38.346639 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4aba78-a16a-496d-8a43-72daf6c6f1bc admin admin] PATH_INFO: `/v3/projects/ac8479cb45c24a0cb187b8878d40a17d/users/3f7dacdc22164308a86b1b197a375b91/roles/136cdf78e00a426997dd4e37faff489b` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:29:38.361947 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4aba78-a16a-496d-8a43-72daf6c6f1bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac8479cb45c24a0cb187b8878d40a17d, user_id=3f7dacdc22164308a86b1b197a375b91, role_id=136cdf78e00a426997dd4e37faff489b)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:29:38.364620 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4aba78-a16a-496d-8a43-72daf6c6f1bc admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:29:38.437698 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1034/2058] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 24 09:29:38 2022] PUT /identity/v3/projects/ac8479cb45c24a0cb187b8878d40a17d/users/3f7dacdc22164308a86b1b197a375b91/roles/136cdf78e00a426997dd4e37faff489b => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:29:38.448367 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2a44d2-f1c6-4c16-a3bf-9af044aa0a3e None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:29:38.448656 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2a44d2-f1c6-4c16-a3bf-9af044aa0a3e None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:29:38.449005 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2a44d2-f1c6-4c16-a3bf-9af044aa0a3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:29:38.477537 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-8a2a44d2-f1c6-4c16-a3bf-9af044aa0a3e None None] MFA Rules not processed for user `3f7dacdc22164308a86b1b197a375b91`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:29:38.478748 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-8a2a44d2-f1c6-4c16-a3bf-9af044aa0a3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:29:38.526414 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:29:38.544135 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1025/2059] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 09:29:38 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 09:29:38.548733 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4aba78-a16a-496d-8a43-72daf6c6f1bc admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:29:38.622399 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4aba78-a16a-496d-8a43-72daf6c6f1bc admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:29:38.627106 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a30e8ec-001e-44fc-9ec8-2777d5ed0cc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:29:38.627748 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a30e8ec-001e-44fc-9ec8-2777d5ed0cc7 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:29:38.628031 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a30e8ec-001e-44fc-9ec8-2777d5ed0cc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:29:38.628304 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a30e8ec-001e-44fc-9ec8-2777d5ed0cc7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:29:38.628930 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a30e8ec-001e-44fc-9ec8-2777d5ed0cc7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:29:38.630827 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a30e8ec-001e-44fc-9ec8-2777d5ed0cc7 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:29:38.719920 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1035/2060] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 24 09:29:38 2022] POST /identity/v3/projects => generated 385 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:29:38.724121 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2a44d2-f1c6-4c16-a3bf-9af044aa0a3e None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:29:38.735920 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2a44d2-f1c6-4c16-a3bf-9af044aa0a3e None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:29:38.799351 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1f659d-4280-4458-83c1-aea6862c2fe0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:29:38.800030 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1f659d-4280-4458-83c1-aea6862c2fe0 admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:29:38.800277 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1f659d-4280-4458-83c1-aea6862c2fe0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:29:38.800536 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1f659d-4280-4458-83c1-aea6862c2fe0 admin admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:29:38.801173 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1f659d-4280-4458-83c1-aea6862c2fe0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:29:38.805142 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1f659d-4280-4458-83c1-aea6862c2fe0 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:29:38.879490 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1026/2061] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 24 09:29:38 2022] POST /identity/v3/users => generated 293 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:29:38.883604 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a30e8ec-001e-44fc-9ec8-2777d5ed0cc7 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:29:38.894829 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a30e8ec-001e-44fc-9ec8-2777d5ed0cc7 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:29:38.899702 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3398bf9f-6e1d-4ecb-bf00-f8d674743a42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:29:38.900316 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3398bf9f-6e1d-4ecb-bf00-f8d674743a42 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:29:38.904590 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3398bf9f-6e1d-4ecb-bf00-f8d674743a42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:29:38.904590 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3398bf9f-6e1d-4ecb-bf00-f8d674743a42 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:29:38.904590 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3398bf9f-6e1d-4ecb-bf00-f8d674743a42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:29:38.904590 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3398bf9f-6e1d-4ecb-bf00-f8d674743a42 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:29:38.915160 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1036/2062] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 24 09:29:38 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:29:38.919746 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1f659d-4280-4458-83c1-aea6862c2fe0 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:29:38.930865 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1f659d-4280-4458-83c1-aea6862c2fe0 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:29:38.936805 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7adb507-cf1c-4af6-94aa-29413cb7a9eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:29:38.937658 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7adb507-cf1c-4af6-94aa-29413cb7a9eb admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:29:38.938041 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7adb507-cf1c-4af6-94aa-29413cb7a9eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:29:38.938402 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7adb507-cf1c-4af6-94aa-29413cb7a9eb admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:29:38.939172 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7adb507-cf1c-4af6-94aa-29413cb7a9eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:29:38.941778 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7adb507-cf1c-4af6-94aa-29413cb7a9eb admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:29:38.959603 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1027/2063] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 24 09:29:38 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:29:38.964233 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3398bf9f-6e1d-4ecb-bf00-f8d674743a42 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:29:38.974455 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3398bf9f-6e1d-4ecb-bf00-f8d674743a42 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:29:38.978975 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4daf8b-55b6-4a3c-83fd-9a3a90752568 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:29:38.979650 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4daf8b-55b6-4a3c-83fd-9a3a90752568 admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:29:38.980460 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4daf8b-55b6-4a3c-83fd-9a3a90752568 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:29:38.980732 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4daf8b-55b6-4a3c-83fd-9a3a90752568 admin admin] PATH_INFO: `/v3/projects/a208fdcbb0a14d84ae2e8940cb716721/users/e1799d7540e244a0a7c7c4a9ab72cef8/roles/136cdf78e00a426997dd4e37faff489b` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:29:38.994636 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4daf8b-55b6-4a3c-83fd-9a3a90752568 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a208fdcbb0a14d84ae2e8940cb716721, user_id=e1799d7540e244a0a7c7c4a9ab72cef8, role_id=136cdf78e00a426997dd4e37faff489b)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:29:38.997944 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4daf8b-55b6-4a3c-83fd-9a3a90752568 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:29:39.071936 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1037/2064] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 24 09:29:38 2022] PUT /identity/v3/projects/a208fdcbb0a14d84ae2e8940cb716721/users/e1799d7540e244a0a7c7c4a9ab72cef8/roles/136cdf78e00a426997dd4e37faff489b => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 09:29:39.082287 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af45348-1f3c-45e2-95d1-9151b098dff3 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:29:39.082574 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af45348-1f3c-45e2-95d1-9151b098dff3 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:29:39.082846 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af45348-1f3c-45e2-95d1-9151b098dff3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:29:39.111202 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-5af45348-1f3c-45e2-95d1-9151b098dff3 None None] MFA Rules not processed for user `e1799d7540e244a0a7c7c4a9ab72cef8`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:29:39.112448 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-5af45348-1f3c-45e2-95d1-9151b098dff3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:29:39.177996 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1028/2065] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 09:29:39 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 09:29:39.182217 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4daf8b-55b6-4a3c-83fd-9a3a90752568 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:29:39.249605 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4daf8b-55b6-4a3c-83fd-9a3a90752568 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:29:39.256001 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9a40c3-fdd8-427d-8afd-9fe7f87dfa73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:29:39.256607 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9a40c3-fdd8-427d-8afd-9fe7f87dfa73 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:29:39.256892 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9a40c3-fdd8-427d-8afd-9fe7f87dfa73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:29:39.257167 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9a40c3-fdd8-427d-8afd-9fe7f87dfa73 admin admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:29:39.257957 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9a40c3-fdd8-427d-8afd-9fe7f87dfa73 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:29:39.259790 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9a40c3-fdd8-427d-8afd-9fe7f87dfa73 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:29:39.346117 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:29:39.353492 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1038/2066] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 24 09:29:39 2022] POST /identity/v3/projects => generated 383 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:29:39.358332 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af45348-1f3c-45e2-95d1-9151b098dff3 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:29:39.371405 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af45348-1f3c-45e2-95d1-9151b098dff3 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:29:39.434985 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f061bb87-dfb9-4830-b06e-5a45c28bbd2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:29:39.435686 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f061bb87-dfb9-4830-b06e-5a45c28bbd2d admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:29:39.436016 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f061bb87-dfb9-4830-b06e-5a45c28bbd2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:29:39.436252 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f061bb87-dfb9-4830-b06e-5a45c28bbd2d admin admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:29:39.436883 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f061bb87-dfb9-4830-b06e-5a45c28bbd2d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:29:39.438803 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f061bb87-dfb9-4830-b06e-5a45c28bbd2d admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:29:39.447177 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:29:39.535445 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1029/2067] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 24 09:29:39 2022] POST /identity/v3/users => generated 298 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 09:29:39.539306 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9a40c3-fdd8-427d-8afd-9fe7f87dfa73 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:29:39.549960 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9a40c3-fdd8-427d-8afd-9fe7f87dfa73 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:29:39.554600 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5c1398-cc1c-477a-8b13-be5575420c46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:29:39.555197 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5c1398-cc1c-477a-8b13-be5575420c46 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:29:39.555474 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5c1398-cc1c-477a-8b13-be5575420c46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:29:39.555780 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5c1398-cc1c-477a-8b13-be5575420c46 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:29:39.556421 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5c1398-cc1c-477a-8b13-be5575420c46 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:29:39.558031 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5c1398-cc1c-477a-8b13-be5575420c46 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:29:39.568446 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1039/2068] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 24 09:29:39 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:29:39.573248 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f061bb87-dfb9-4830-b06e-5a45c28bbd2d admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:29:39.585084 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f061bb87-dfb9-4830-b06e-5a45c28bbd2d admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:29:39.592820 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b157c2d6-3947-4176-8833-8179d4023ae4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:29:39.593517 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b157c2d6-3947-4176-8833-8179d4023ae4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:29:39.593796 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b157c2d6-3947-4176-8833-8179d4023ae4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:29:39.594068 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b157c2d6-3947-4176-8833-8179d4023ae4 admin admin] PATH_INFO: `/v3/projects/31ab22450c5146c99fda3aafa7770a98/users/263e600bea4840efa19b0510e57f2cc4/roles/15f634d4a2d247ddab90c458e5e1c141` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:29:39.609509 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b157c2d6-3947-4176-8833-8179d4023ae4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=31ab22450c5146c99fda3aafa7770a98, user_id=263e600bea4840efa19b0510e57f2cc4, role_id=15f634d4a2d247ddab90c458e5e1c141)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:29:39.612635 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b157c2d6-3947-4176-8833-8179d4023ae4 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:29:39.701472 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1030/2069] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 24 09:29:39 2022] PUT /identity/v3/projects/31ab22450c5146c99fda3aafa7770a98/users/263e600bea4840efa19b0510e57f2cc4/roles/15f634d4a2d247ddab90c458e5e1c141 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:29:39.706177 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5c1398-cc1c-477a-8b13-be5575420c46 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:29:39.718211 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5c1398-cc1c-477a-8b13-be5575420c46 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:29:39.775706 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4954712e-e209-41c5-86f3-8a9f430b0be8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:29:39.776313 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4954712e-e209-41c5-86f3-8a9f430b0be8 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:29:39.776594 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4954712e-e209-41c5-86f3-8a9f430b0be8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:29:39.777630 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4954712e-e209-41c5-86f3-8a9f430b0be8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:29:39.778160 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4954712e-e209-41c5-86f3-8a9f430b0be8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:29:39.779774 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4954712e-e209-41c5-86f3-8a9f430b0be8 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:29:39.789533 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1040/2070] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 24 09:29:39 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:29:39.794626 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b157c2d6-3947-4176-8833-8179d4023ae4 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:29:39.808058 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b157c2d6-3947-4176-8833-8179d4023ae4 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:29:39.813069 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf24bb84-1647-4ae8-a5fd-98d3d0f46e20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:29:39.813738 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf24bb84-1647-4ae8-a5fd-98d3d0f46e20 admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:29:39.814017 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf24bb84-1647-4ae8-a5fd-98d3d0f46e20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:29:39.814288 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf24bb84-1647-4ae8-a5fd-98d3d0f46e20 admin admin] PATH_INFO: `/v3/projects/31ab22450c5146c99fda3aafa7770a98/users/263e600bea4840efa19b0510e57f2cc4/roles/15f634d4a2d247ddab90c458e5e1c141` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:29:39.816527 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf24bb84-1647-4ae8-a5fd-98d3d0f46e20 admin admin] RBAC: Authorizing `identity:create_grant(project_id=31ab22450c5146c99fda3aafa7770a98, user_id=263e600bea4840efa19b0510e57f2cc4, role_id=15f634d4a2d247ddab90c458e5e1c141)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:29:39.819625 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf24bb84-1647-4ae8-a5fd-98d3d0f46e20 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:29:39.832204 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1031/2071] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 24 09:29:39 2022] PUT /identity/v3/projects/31ab22450c5146c99fda3aafa7770a98/users/263e600bea4840efa19b0510e57f2cc4/roles/15f634d4a2d247ddab90c458e5e1c141 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:29:39.844212 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5da69c-cf58-4cda-9c0b-db749dc6ce8a None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:29:39.844600 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5da69c-cf58-4cda-9c0b-db749dc6ce8a None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:29:39.844879 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5da69c-cf58-4cda-9c0b-db749dc6ce8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:29:39.877382 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-fa5da69c-cf58-4cda-9c0b-db749dc6ce8a None None] MFA Rules not processed for user `263e600bea4840efa19b0510e57f2cc4`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:29:39.878815 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-fa5da69c-cf58-4cda-9c0b-db749dc6ce8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:29:39.959800 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1041/2072] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 09:29:39 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:29:39.972077 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf24bb84-1647-4ae8-a5fd-98d3d0f46e20 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:29:39.973896 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-bf24bb84-1647-4ae8-a5fd-98d3d0f46e20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:29:40.048569 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf24bb84-1647-4ae8-a5fd-98d3d0f46e20 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:29:40.052202 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712c52a4-59d1-49ed-8821-f492ed318a37 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c32b83577c46488906fe409a790b24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:29:40.052819 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712c52a4-59d1-49ed-8821-f492ed318a37 service cinder] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:29:40.053101 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712c52a4-59d1-49ed-8821-f492ed318a37 service cinder] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:29:40.053368 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712c52a4-59d1-49ed-8821-f492ed318a37 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:29:40.116021 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712c52a4-59d1-49ed-8821-f492ed318a37 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:29:40.117450 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712c52a4-59d1-49ed-8821-f492ed318a37 service cinder] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:29:40.126700 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1032/2073] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:29:39 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:29:43.200702 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5da69c-cf58-4cda-9c0b-db749dc6ce8a None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:29:43.210509 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5da69c-cf58-4cda-9c0b-db749dc6ce8a None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:29:43.214464 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc88b779-d869-49ac-a304-7b0f1e1eaa38 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c32b83577c46488906fe409a790b24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:29:43.215139 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc88b779-d869-49ac-a304-7b0f1e1eaa38 service cinder] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:29:43.215423 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc88b779-d869-49ac-a304-7b0f1e1eaa38 service cinder] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:29:43.215846 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc88b779-d869-49ac-a304-7b0f1e1eaa38 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:29:43.275956 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc88b779-d869-49ac-a304-7b0f1e1eaa38 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:29:43.277433 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc88b779-d869-49ac-a304-7b0f1e1eaa38 service cinder] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:29:43.287386 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1042/2074] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:29:43 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:29:44.741177 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712c52a4-59d1-49ed-8821-f492ed318a37 service cinder] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:29:44.759629 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712c52a4-59d1-49ed-8821-f492ed318a37 service cinder] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:29:44.767759 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de43d50-a25d-4f09-aa8d-5dbd0a35735c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c32b83577c46488906fe409a790b24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:29:44.768633 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de43d50-a25d-4f09-aa8d-5dbd0a35735c service cinder] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:29:44.768913 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de43d50-a25d-4f09-aa8d-5dbd0a35735c service cinder] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:29:44.769192 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de43d50-a25d-4f09-aa8d-5dbd0a35735c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:29:44.779034 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de43d50-a25d-4f09-aa8d-5dbd0a35735c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:29:44.780487 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de43d50-a25d-4f09-aa8d-5dbd0a35735c service cinder] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:29:44.792067 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1033/2075] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:29:44 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:29:47.452986 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989c4159-303f-4f79-8b42-e4c72c07f7ed None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:29:47.453537 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989c4159-303f-4f79-8b42-e4c72c07f7ed None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:29:47.453638 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989c4159-303f-4f79-8b42-e4c72c07f7ed None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:29:47.455043 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1043/2076] 10.222.0.16 () {54 vars in 921 bytes} [Wed Aug 24 09:29:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:29:47.497611 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9a627f-3c71-4ed5-8201-c9850047c8fd None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:29:47.498250 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9a627f-3c71-4ed5-8201-c9850047c8fd None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:29:47.498348 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9a627f-3c71-4ed5-8201-c9850047c8fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:29:47.524406 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-fe9a627f-3c71-4ed5-8201-c9850047c8fd None None] MFA Rules not processed for user `80cada22039d4f11935412ee578acd05`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:29:47.526320 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-fe9a627f-3c71-4ed5-8201-c9850047c8fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:29:47.584860 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:29:47.592279 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989c4159-303f-4f79-8b42-e4c72c07f7ed None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:29:47.596091 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-989c4159-303f-4f79-8b42-e4c72c07f7ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:29:47.609579 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1034/2077] 10.222.0.16 () {58 vars in 1018 bytes} [Wed Aug 24 09:29:47 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:29:47.611851 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:29:47.621574 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9a627f-3c71-4ed5-8201-c9850047c8fd None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:29:47.623359 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-fe9a627f-3c71-4ed5-8201-c9850047c8fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:29:47.637336 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9a627f-3c71-4ed5-8201-c9850047c8fd None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:29:47.642655 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc06690-127e-4fe4-88b9-e7a1243d23f7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80cada22039d4f11935412ee578acd05', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:29:47.643269 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc06690-127e-4fe4-88b9-e7a1243d23f7 service placement] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:29:47.643556 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc06690-127e-4fe4-88b9-e7a1243d23f7 service placement] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:29:47.643887 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc06690-127e-4fe4-88b9-e7a1243d23f7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:29:47.653101 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc06690-127e-4fe4-88b9-e7a1243d23f7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:29:47.654741 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc06690-127e-4fe4-88b9-e7a1243d23f7 service placement] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:29:47.665475 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1035/2078] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:29:47 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 09:29:47.679909 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989c4159-303f-4f79-8b42-e4c72c07f7ed None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:29:47.685168 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de55e3ab-28df-4427-988d-c1c82d1aa67d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:29:47.686025 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de55e3ab-28df-4427-988d-c1c82d1aa67d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:29:47.686315 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de55e3ab-28df-4427-988d-c1c82d1aa67d admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:29:47.686646 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de55e3ab-28df-4427-988d-c1c82d1aa67d admin admin] PATH_INFO: `/v3/users/6cad69b6035948fe91e9beb66c807d28` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:29:47.687949 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de55e3ab-28df-4427-988d-c1c82d1aa67d admin admin] RBAC: Authorizing `identity:delete_user(user_id=6cad69b6035948fe91e9beb66c807d28)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:29:47.690561 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de55e3ab-28df-4427-988d-c1c82d1aa67d admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:29:47.724621 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7425e99b-060a-43c6-b95d-05c6002e2dcc None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:29:47.724908 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7425e99b-060a-43c6-b95d-05c6002e2dcc None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:29:47.725207 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7425e99b-060a-43c6-b95d-05c6002e2dcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:29:47.745700 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-7425e99b-060a-43c6-b95d-05c6002e2dcc None None] MFA Rules not processed for user `b3cd284afb854b1e987e807127752925`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:29:47.747430 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-7425e99b-060a-43c6-b95d-05c6002e2dcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:29:47.826864 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1036/2079] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 09:29:47 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 09:29:47.882443 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-de55e3ab-28df-4427-988d-c1c82d1aa67d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6cad69b6035948fe91e9beb66c807d28'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:29:47.883104 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-de55e3ab-28df-4427-988d-c1c82d1aa67d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6cad69b6035948fe91e9beb66c807d28'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:29:47.940275 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-de55e3ab-28df-4427-988d-c1c82d1aa67d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6cad69b6035948fe91e9beb66c807d28'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:29:47.941006 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-de55e3ab-28df-4427-988d-c1c82d1aa67d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6cad69b6035948fe91e9beb66c807d28'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:29:47.950934 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-de55e3ab-28df-4427-988d-c1c82d1aa67d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6cad69b6035948fe91e9beb66c807d28'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:29:47.969327 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1044/2080] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 24 09:29:47 2022] DELETE /identity/v3/users/6cad69b6035948fe91e9beb66c807d28 => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:29:48.217850 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7425e99b-060a-43c6-b95d-05c6002e2dcc None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:29:48.235932 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7425e99b-060a-43c6-b95d-05c6002e2dcc None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:29:48.304926 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8f930c-adb5-4c30-85c3-2437373876c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:29:48.305387 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8f930c-adb5-4c30-85c3-2437373876c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:29:48.305665 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8f930c-adb5-4c30-85c3-2437373876c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:29:48.305937 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8f930c-adb5-4c30-85c3-2437373876c4 admin admin] PATH_INFO: `/v3/projects/fe8949d7fc8b4ca89fd81421d7e94122` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:29:48.307063 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8f930c-adb5-4c30-85c3-2437373876c4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fe8949d7fc8b4ca89fd81421d7e94122)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:29:48.309539 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8f930c-adb5-4c30-85c3-2437373876c4 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:29:48.431730 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-6b8f930c-adb5-4c30-85c3-2437373876c4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fe8949d7fc8b4ca89fd81421d7e94122', 'request_id': 'req-6b8f930c-adb5-4c30-85c3-2437373876c4'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:29:48.438623 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-6b8f930c-adb5-4c30-85c3-2437373876c4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fe8949d7fc8b4ca89fd81421d7e94122', 'request_id': 'req-6b8f930c-adb5-4c30-85c3-2437373876c4'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:29:48.445196 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-6b8f930c-adb5-4c30-85c3-2437373876c4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fe8949d7fc8b4ca89fd81421d7e94122', 'request_id': 'req-6b8f930c-adb5-4c30-85c3-2437373876c4'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:29:48.488373 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-6b8f930c-adb5-4c30-85c3-2437373876c4 admin admin] The token cache is being invalidate because project fe8949d7fc8b4ca89fd81421d7e94122 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:29:48.490528 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-6b8f930c-adb5-4c30-85c3-2437373876c4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:29:48.490528 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-6b8f930c-adb5-4c30-85c3-2437373876c4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:29:48.490846 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1037/2081] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 24 09:29:48 2022] DELETE /identity/v3/projects/fe8949d7fc8b4ca89fd81421d7e94122 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:29:48.496015 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8d884d-6b81-401a-88b1-457764302457 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:29:48.497140 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8d884d-6b81-401a-88b1-457764302457 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:29:48.497311 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8d884d-6b81-401a-88b1-457764302457 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:29:48.518013 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-1a8d884d-6b81-401a-88b1-457764302457 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:29:48.519368 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-1a8d884d-6b81-401a-88b1-457764302457 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:29:48.590221 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1045/2082] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 09:29:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:29:48.599347 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2bda8e-1a37-4e60-85ca-bd511d575b7c None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:29:48.599928 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2bda8e-1a37-4e60-85ca-bd511d575b7c None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:29:48.599928 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2bda8e-1a37-4e60-85ca-bd511d575b7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:29:48.628578 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-ea2bda8e-1a37-4e60-85ca-bd511d575b7c None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:29:48.629873 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-ea2bda8e-1a37-4e60-85ca-bd511d575b7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:29:48.641578 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1038/2083] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 09:29:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:29:48.647147 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8d884d-6b81-401a-88b1-457764302457 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:29:48.657476 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8d884d-6b81-401a-88b1-457764302457 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:29:48.662529 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8673a0c6-9e5a-4db0-9eb6-3ee0752dad95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:29:48.663122 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8673a0c6-9e5a-4db0-9eb6-3ee0752dad95 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:29:48.663409 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8673a0c6-9e5a-4db0-9eb6-3ee0752dad95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:29:48.663749 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8673a0c6-9e5a-4db0-9eb6-3ee0752dad95 admin admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:29:48.664439 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8673a0c6-9e5a-4db0-9eb6-3ee0752dad95 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:29:48.664748 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8673a0c6-9e5a-4db0-9eb6-3ee0752dad95 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:29:48.666560 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8673a0c6-9e5a-4db0-9eb6-3ee0752dad95 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:29:48.679896 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1046/2084] 10.222.0.16 () {58 vars in 1153 bytes} [Wed Aug 24 09:29:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:29:48.685266 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2bda8e-1a37-4e60-85ca-bd511d575b7c None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:29:48.692390 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:29:48.707938 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2bda8e-1a37-4e60-85ca-bd511d575b7c None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:29:48.713581 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dba8008-5726-4dcd-9d2d-1a1b9d5d429c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:29:48.714222 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dba8008-5726-4dcd-9d2d-1a1b9d5d429c admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:29:48.714533 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dba8008-5726-4dcd-9d2d-1a1b9d5d429c admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:29:48.714823 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dba8008-5726-4dcd-9d2d-1a1b9d5d429c admin admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:29:48.715433 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dba8008-5726-4dcd-9d2d-1a1b9d5d429c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:29:48.726394 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dba8008-5726-4dcd-9d2d-1a1b9d5d429c admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:29:48.813038 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1039/2085] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 24 09:29:48 2022] POST /identity/v3/projects => generated 393 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:29:48.817271 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8673a0c6-9e5a-4db0-9eb6-3ee0752dad95 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:29:48.829143 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8673a0c6-9e5a-4db0-9eb6-3ee0752dad95 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:29:48.839479 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:29:48.913713 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdf27f8-fc7d-4211-8388-d2ee816e38d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:29:48.914112 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdf27f8-fc7d-4211-8388-d2ee816e38d0 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:29:48.914555 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdf27f8-fc7d-4211-8388-d2ee816e38d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:29:48.914829 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdf27f8-fc7d-4211-8388-d2ee816e38d0 admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:29:48.915455 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdf27f8-fc7d-4211-8388-d2ee816e38d0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:29:48.917795 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdf27f8-fc7d-4211-8388-d2ee816e38d0 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:29:49.041334 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1047/2086] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 24 09:29:48 2022] POST /identity/v3/users => generated 297 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:29:49.046022 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dba8008-5726-4dcd-9d2d-1a1b9d5d429c admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:29:49.057955 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dba8008-5726-4dcd-9d2d-1a1b9d5d429c admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:29:49.064124 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83702c7b-ecae-4447-b765-3d9d6f42f50d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:29:49.064894 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83702c7b-ecae-4447-b765-3d9d6f42f50d admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:29:49.070084 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83702c7b-ecae-4447-b765-3d9d6f42f50d admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:29:49.070084 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83702c7b-ecae-4447-b765-3d9d6f42f50d admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:29:49.070084 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83702c7b-ecae-4447-b765-3d9d6f42f50d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:29:49.070084 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83702c7b-ecae-4447-b765-3d9d6f42f50d admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:29:49.080652 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1040/2087] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 24 09:29:49 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:29:49.085308 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdf27f8-fc7d-4211-8388-d2ee816e38d0 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:29:49.105360 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdf27f8-fc7d-4211-8388-d2ee816e38d0 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:29:49.110872 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab34fb6-a628-4400-ad2d-28d6ff5efc8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:29:49.111486 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab34fb6-a628-4400-ad2d-28d6ff5efc8e admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:29:49.111863 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab34fb6-a628-4400-ad2d-28d6ff5efc8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:29:49.112176 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab34fb6-a628-4400-ad2d-28d6ff5efc8e admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:29:49.112823 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab34fb6-a628-4400-ad2d-28d6ff5efc8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:29:49.114426 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab34fb6-a628-4400-ad2d-28d6ff5efc8e admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:29:49.125468 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1048/2088] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 24 09:29:49 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:29:49.129188 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83702c7b-ecae-4447-b765-3d9d6f42f50d admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:29:49.139421 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83702c7b-ecae-4447-b765-3d9d6f42f50d admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:29:49.144847 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d891d420-44dd-4b8b-92d3-b1856c18e69a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:29:49.144847 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d891d420-44dd-4b8b-92d3-b1856c18e69a admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:29:49.145058 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d891d420-44dd-4b8b-92d3-b1856c18e69a admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:29:49.145270 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d891d420-44dd-4b8b-92d3-b1856c18e69a admin admin] PATH_INFO: `/v3/projects/7a0a723d97d24385875a287252f57f02/users/0aff36cf71984238b8a9b998043ec7be/roles/136cdf78e00a426997dd4e37faff489b` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:29:49.167680 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d891d420-44dd-4b8b-92d3-b1856c18e69a admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a0a723d97d24385875a287252f57f02, user_id=0aff36cf71984238b8a9b998043ec7be, role_id=136cdf78e00a426997dd4e37faff489b)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:29:49.171006 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d891d420-44dd-4b8b-92d3-b1856c18e69a admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:29:49.295198 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1041/2089] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 24 09:29:49 2022] PUT /identity/v3/projects/7a0a723d97d24385875a287252f57f02/users/0aff36cf71984238b8a9b998043ec7be/roles/136cdf78e00a426997dd4e37faff489b => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:29:49.300418 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be1935e-1b07-4124-8f42-56ef29a4d4fd None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:29:49.300953 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be1935e-1b07-4124-8f42-56ef29a4d4fd None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:29:49.301236 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be1935e-1b07-4124-8f42-56ef29a4d4fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:29:49.321652 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-7be1935e-1b07-4124-8f42-56ef29a4d4fd None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:29:49.322916 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-7be1935e-1b07-4124-8f42-56ef29a4d4fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:29:49.394989 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1049/2090] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 09:29:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 09:29:49.405457 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d891d420-44dd-4b8b-92d3-b1856c18e69a admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:29:49.407203 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-d891d420-44dd-4b8b-92d3-b1856c18e69a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:29:49.466498 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d891d420-44dd-4b8b-92d3-b1856c18e69a admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:29:49.470475 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f00bd8-9906-4850-b7a2-a4e516c7e1d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e5260e798404e8bb12b25ca8ea97052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:29:49.471426 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f00bd8-9906-4850-b7a2-a4e516c7e1d6 service neutron] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:29:49.471752 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f00bd8-9906-4850-b7a2-a4e516c7e1d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:29:49.474904 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f00bd8-9906-4850-b7a2-a4e516c7e1d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:29:49.481785 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f00bd8-9906-4850-b7a2-a4e516c7e1d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:29:49.483266 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f00bd8-9906-4850-b7a2-a4e516c7e1d6 service neutron] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:29:49.496196 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1042/2091] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:29:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:29:51.775708 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be1935e-1b07-4124-8f42-56ef29a4d4fd None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:29:51.777567 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-7be1935e-1b07-4124-8f42-56ef29a4d4fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:29:51.793435 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be1935e-1b07-4124-8f42-56ef29a4d4fd None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:29:51.798169 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fde30c8-55a4-4500-9099-5dc01f0e5727 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:29:51.798879 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fde30c8-55a4-4500-9099-5dc01f0e5727 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:29:51.799164 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fde30c8-55a4-4500-9099-5dc01f0e5727 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:29:51.799435 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fde30c8-55a4-4500-9099-5dc01f0e5727 admin admin] PATH_INFO: `/v3/users/3f7dacdc22164308a86b1b197a375b91` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:29:51.800715 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fde30c8-55a4-4500-9099-5dc01f0e5727 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3f7dacdc22164308a86b1b197a375b91)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:29:51.802660 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fde30c8-55a4-4500-9099-5dc01f0e5727 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:29:52.107710 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-1fde30c8-55a4-4500-9099-5dc01f0e5727 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3f7dacdc22164308a86b1b197a375b91'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:29:52.112689 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-1fde30c8-55a4-4500-9099-5dc01f0e5727 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3f7dacdc22164308a86b1b197a375b91'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:29:52.162960 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-1fde30c8-55a4-4500-9099-5dc01f0e5727 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3f7dacdc22164308a86b1b197a375b91'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:29:52.163871 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-1fde30c8-55a4-4500-9099-5dc01f0e5727 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3f7dacdc22164308a86b1b197a375b91'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:29:52.176253 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-1fde30c8-55a4-4500-9099-5dc01f0e5727 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3f7dacdc22164308a86b1b197a375b91'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:29:52.203367 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1050/2092] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 24 09:29:51 2022] DELETE /identity/v3/users/3f7dacdc22164308a86b1b197a375b91 => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:29:52.204779 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcea634-36c0-42ad-9f2d-460848a8ec71 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:29:52.205240 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcea634-36c0-42ad-9f2d-460848a8ec71 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:29:52.205349 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcea634-36c0-42ad-9f2d-460848a8ec71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:29:52.237413 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-9bcea634-36c0-42ad-9f2d-460848a8ec71 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:29:52.239203 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-9bcea634-36c0-42ad-9f2d-460848a8ec71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:29:52.340611 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1043/2093] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 09:29:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 09:29:52.351842 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fde30c8-55a4-4500-9099-5dc01f0e5727 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:29:52.353389 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-1fde30c8-55a4-4500-9099-5dc01f0e5727 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:29:52.360782 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:29:52.448269 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fde30c8-55a4-4500-9099-5dc01f0e5727 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:29:52.452495 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b269782e-fbff-4135-809c-17759fa0e6e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e5260e798404e8bb12b25ca8ea97052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:29:52.452908 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b269782e-fbff-4135-809c-17759fa0e6e8 service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:29:52.453198 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b269782e-fbff-4135-809c-17759fa0e6e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:29:52.453522 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b269782e-fbff-4135-809c-17759fa0e6e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:29:52.464514 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b269782e-fbff-4135-809c-17759fa0e6e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:29:52.468385 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b269782e-fbff-4135-809c-17759fa0e6e8 service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:29:52.479913 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1051/2094] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:29:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:29:53.101120 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcea634-36c0-42ad-9f2d-460848a8ec71 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:29:53.102612 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-9bcea634-36c0-42ad-9f2d-460848a8ec71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:29:53.132066 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcea634-36c0-42ad-9f2d-460848a8ec71 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:29:53.141835 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea05863-2a3c-4d30-b11c-c0bbea3f76ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:29:53.142499 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea05863-2a3c-4d30-b11c-c0bbea3f76ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:29:53.142787 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea05863-2a3c-4d30-b11c-c0bbea3f76ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:29:53.143070 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea05863-2a3c-4d30-b11c-c0bbea3f76ab admin admin] PATH_INFO: `/v3/projects/ac8479cb45c24a0cb187b8878d40a17d` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:29:53.144416 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea05863-2a3c-4d30-b11c-c0bbea3f76ab admin admin] RBAC: Authorizing `identity:delete_project(project_id=ac8479cb45c24a0cb187b8878d40a17d)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:29:53.146532 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea05863-2a3c-4d30-b11c-c0bbea3f76ab admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:29:53.309353 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:29:53.312753 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-aea05863-2a3c-4d30-b11c-c0bbea3f76ab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ac8479cb45c24a0cb187b8878d40a17d', 'request_id': 'req-aea05863-2a3c-4d30-b11c-c0bbea3f76ab'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:29:53.322604 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-aea05863-2a3c-4d30-b11c-c0bbea3f76ab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ac8479cb45c24a0cb187b8878d40a17d', 'request_id': 'req-aea05863-2a3c-4d30-b11c-c0bbea3f76ab'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:29:53.329875 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-aea05863-2a3c-4d30-b11c-c0bbea3f76ab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ac8479cb45c24a0cb187b8878d40a17d', 'request_id': 'req-aea05863-2a3c-4d30-b11c-c0bbea3f76ab'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:29:53.427359 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-aea05863-2a3c-4d30-b11c-c0bbea3f76ab admin admin] The token cache is being invalidate because project ac8479cb45c24a0cb187b8878d40a17d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:29:53.430719 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-aea05863-2a3c-4d30-b11c-c0bbea3f76ab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:29:53.430719 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-aea05863-2a3c-4d30-b11c-c0bbea3f76ab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:29:53.430956 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1044/2095] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 24 09:29:53 2022] DELETE /identity/v3/projects/ac8479cb45c24a0cb187b8878d40a17d => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 09:29:53.436159 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b269782e-fbff-4135-809c-17759fa0e6e8 service neutron] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:29:53.446698 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b269782e-fbff-4135-809c-17759fa0e6e8 service neutron] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:29:53.555524 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffc5cb2-836e-4ea7-9991-58e20113e4b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:29:53.567559 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffc5cb2-836e-4ea7-9991-58e20113e4b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:29:53.567559 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffc5cb2-836e-4ea7-9991-58e20113e4b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:29:53.567559 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffc5cb2-836e-4ea7-9991-58e20113e4b3 admin admin] PATH_INFO: `/v3/users/e1799d7540e244a0a7c7c4a9ab72cef8` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:29:53.567559 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffc5cb2-836e-4ea7-9991-58e20113e4b3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e1799d7540e244a0a7c7c4a9ab72cef8)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:29:53.567559 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffc5cb2-836e-4ea7-9991-58e20113e4b3 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:29:53.800190 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-fffc5cb2-836e-4ea7-9991-58e20113e4b3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e1799d7540e244a0a7c7c4a9ab72cef8'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:29:53.801238 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-fffc5cb2-836e-4ea7-9991-58e20113e4b3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e1799d7540e244a0a7c7c4a9ab72cef8'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:29:53.904661 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-fffc5cb2-836e-4ea7-9991-58e20113e4b3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e1799d7540e244a0a7c7c4a9ab72cef8'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:29:53.905652 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-fffc5cb2-836e-4ea7-9991-58e20113e4b3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e1799d7540e244a0a7c7c4a9ab72cef8'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:29:53.917180 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-fffc5cb2-836e-4ea7-9991-58e20113e4b3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e1799d7540e244a0a7c7c4a9ab72cef8'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:29:53.938923 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1052/2096] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 24 09:29:53 2022] DELETE /identity/v3/users/e1799d7540e244a0a7c7c4a9ab72cef8 => generated 0 bytes in 503 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:29:54.608526 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea05863-2a3c-4d30-b11c-c0bbea3f76ab admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:29:54.609973 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-aea05863-2a3c-4d30-b11c-c0bbea3f76ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:29:54.640989 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea05863-2a3c-4d30-b11c-c0bbea3f76ab admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:29:54.726890 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919390be-4c32-4cd8-bea3-1c88b2367244 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:29:54.727568 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919390be-4c32-4cd8-bea3-1c88b2367244 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:29:54.728396 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919390be-4c32-4cd8-bea3-1c88b2367244 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:29:54.728396 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919390be-4c32-4cd8-bea3-1c88b2367244 admin admin] PATH_INFO: `/v3/projects/a208fdcbb0a14d84ae2e8940cb716721` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:29:54.731597 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919390be-4c32-4cd8-bea3-1c88b2367244 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a208fdcbb0a14d84ae2e8940cb716721)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:29:54.733212 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919390be-4c32-4cd8-bea3-1c88b2367244 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:29:54.884132 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-919390be-4c32-4cd8-bea3-1c88b2367244 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a208fdcbb0a14d84ae2e8940cb716721', 'request_id': 'req-919390be-4c32-4cd8-bea3-1c88b2367244'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:29:54.891294 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-919390be-4c32-4cd8-bea3-1c88b2367244 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a208fdcbb0a14d84ae2e8940cb716721', 'request_id': 'req-919390be-4c32-4cd8-bea3-1c88b2367244'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:29:54.899552 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-919390be-4c32-4cd8-bea3-1c88b2367244 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a208fdcbb0a14d84ae2e8940cb716721', 'request_id': 'req-919390be-4c32-4cd8-bea3-1c88b2367244'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:29:54.963201 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-919390be-4c32-4cd8-bea3-1c88b2367244 admin admin] The token cache is being invalidate because project a208fdcbb0a14d84ae2e8940cb716721 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:29:54.963702 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-919390be-4c32-4cd8-bea3-1c88b2367244 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:29:54.964958 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-919390be-4c32-4cd8-bea3-1c88b2367244 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:29:54.966953 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1045/2097] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 24 09:29:54 2022] DELETE /identity/v3/projects/a208fdcbb0a14d84ae2e8940cb716721 => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:29:54.970918 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffc5cb2-836e-4ea7-9991-58e20113e4b3 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:29:54.972604 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-fffc5cb2-836e-4ea7-9991-58e20113e4b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:29:54.998534 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffc5cb2-836e-4ea7-9991-58e20113e4b3 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:29:55.038152 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:29:55.084068 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a437a66e-cda6-40df-836e-e6003327ace7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:29:55.085155 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a437a66e-cda6-40df-836e-e6003327ace7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:29:55.085584 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a437a66e-cda6-40df-836e-e6003327ace7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:29:55.086056 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a437a66e-cda6-40df-836e-e6003327ace7 admin admin] PATH_INFO: `/v3/users/263e600bea4840efa19b0510e57f2cc4` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:29:55.088021 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a437a66e-cda6-40df-836e-e6003327ace7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=263e600bea4840efa19b0510e57f2cc4)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:29:55.090154 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a437a66e-cda6-40df-836e-e6003327ace7 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:29:55.314754 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-a437a66e-cda6-40df-836e-e6003327ace7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '263e600bea4840efa19b0510e57f2cc4'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:29:55.315433 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-a437a66e-cda6-40df-836e-e6003327ace7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '263e600bea4840efa19b0510e57f2cc4'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:29:55.414120 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-a437a66e-cda6-40df-836e-e6003327ace7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '263e600bea4840efa19b0510e57f2cc4'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:29:55.418473 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-a437a66e-cda6-40df-836e-e6003327ace7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '263e600bea4840efa19b0510e57f2cc4'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:29:55.423422 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-a437a66e-cda6-40df-836e-e6003327ace7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '263e600bea4840efa19b0510e57f2cc4'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:29:55.460306 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1053/2098] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 24 09:29:54 2022] DELETE /identity/v3/users/263e600bea4840efa19b0510e57f2cc4 => generated 0 bytes in 490 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:29:55.858336 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b08345-e3d9-400c-867f-211aa5008ff7 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:29:55.858840 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b08345-e3d9-400c-867f-211aa5008ff7 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:29:55.858954 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b08345-e3d9-400c-867f-211aa5008ff7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:29:55.897730 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-e6b08345-e3d9-400c-867f-211aa5008ff7 None None] MFA Rules not processed for user `0aff36cf71984238b8a9b998043ec7be`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:29:55.899802 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-e6b08345-e3d9-400c-867f-211aa5008ff7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:29:55.981917 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:DELETE=11,keystone:SELECT=87,keystone:INSERT=2 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:29:55.994092 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1046/2099] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 09:29:55 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:29:56.005968 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a397c9a2-93fd-49c0-86ad-2d9ecd8a67c8 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:29:56.006609 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a397c9a2-93fd-49c0-86ad-2d9ecd8a67c8 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:29:56.006609 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a397c9a2-93fd-49c0-86ad-2d9ecd8a67c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:29:56.026950 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-a397c9a2-93fd-49c0-86ad-2d9ecd8a67c8 None None] MFA Rules not processed for user `0aff36cf71984238b8a9b998043ec7be`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:29:56.028282 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-a397c9a2-93fd-49c0-86ad-2d9ecd8a67c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:29:56.036593 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1054/2100] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 09:29:56 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:29:56.048480 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b08345-e3d9-400c-867f-211aa5008ff7 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:29:56.050466 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-e6b08345-e3d9-400c-867f-211aa5008ff7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:29:56.077091 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a397c9a2-93fd-49c0-86ad-2d9ecd8a67c8 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:29:56.078915 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-a397c9a2-93fd-49c0-86ad-2d9ecd8a67c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:29:56.165867 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b08345-e3d9-400c-867f-211aa5008ff7 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:29:56.175235 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0c0b6c-e7f8-4621-896b-a2c2ccb40ded service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f055f7dc51d43bda9b315ca67f11d58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:29:56.175914 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0c0b6c-e7f8-4621-896b-a2c2ccb40ded service nova] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:29:56.176201 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0c0b6c-e7f8-4621-896b-a2c2ccb40ded service nova] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:29:56.176478 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0c0b6c-e7f8-4621-896b-a2c2ccb40ded service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:29:56.197354 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0c0b6c-e7f8-4621-896b-a2c2ccb40ded service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:29:56.199086 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0c0b6c-e7f8-4621-896b-a2c2ccb40ded service nova] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:29:56.215115 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1047/2101] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:29:56 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:29:56.223865 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a397c9a2-93fd-49c0-86ad-2d9ecd8a67c8 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:29:56.228872 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d235a3-0956-4744-a965-1f88668a770d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:29:56.229783 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d235a3-0956-4744-a965-1f88668a770d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:29:56.230836 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d235a3-0956-4744-a965-1f88668a770d admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:29:56.231299 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d235a3-0956-4744-a965-1f88668a770d admin admin] PATH_INFO: `/v3/projects/31ab22450c5146c99fda3aafa7770a98` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:29:56.232506 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d235a3-0956-4744-a965-1f88668a770d admin admin] RBAC: Authorizing `identity:delete_project(project_id=31ab22450c5146c99fda3aafa7770a98)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:29:56.234381 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d235a3-0956-4744-a965-1f88668a770d admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:29:56.346330 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:29:56.352803 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-66d235a3-0956-4744-a965-1f88668a770d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '31ab22450c5146c99fda3aafa7770a98', 'request_id': 'req-66d235a3-0956-4744-a965-1f88668a770d'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:29:56.359729 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-66d235a3-0956-4744-a965-1f88668a770d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '31ab22450c5146c99fda3aafa7770a98', 'request_id': 'req-66d235a3-0956-4744-a965-1f88668a770d'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:29:56.365374 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-66d235a3-0956-4744-a965-1f88668a770d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '31ab22450c5146c99fda3aafa7770a98', 'request_id': 'req-66d235a3-0956-4744-a965-1f88668a770d'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:29:56.437603 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-66d235a3-0956-4744-a965-1f88668a770d admin admin] The token cache is being invalidate because project 31ab22450c5146c99fda3aafa7770a98 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81602) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:29:56.437903 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-66d235a3-0956-4744-a965-1f88668a770d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:29:56.438570 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-66d235a3-0956-4744-a965-1f88668a770d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:29:56.444897 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1055/2102] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 24 09:29:56 2022] DELETE /identity/v3/projects/31ab22450c5146c99fda3aafa7770a98 => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:29:56.450216 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3743e521-43d2-44d6-8f23-118a211912fb None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:29:56.450520 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3743e521-43d2-44d6-8f23-118a211912fb None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:29:56.450852 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3743e521-43d2-44d6-8f23-118a211912fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:29:56.476732 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-3743e521-43d2-44d6-8f23-118a211912fb None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:29:56.477955 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-3743e521-43d2-44d6-8f23-118a211912fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:29:56.544539 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1048/2103] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 09:29:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:29:56.552582 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b0b95d-2c32-4add-be48-dac6b4809a87 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:29:56.552896 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b0b95d-2c32-4add-be48-dac6b4809a87 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:29:56.553175 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b0b95d-2c32-4add-be48-dac6b4809a87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:29:56.573733 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-d8b0b95d-2c32-4add-be48-dac6b4809a87 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:29:56.574992 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-d8b0b95d-2c32-4add-be48-dac6b4809a87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:29:56.583132 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1056/2104] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 09:29:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:29:56.587748 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3743e521-43d2-44d6-8f23-118a211912fb None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:29:56.598045 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3743e521-43d2-44d6-8f23-118a211912fb None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:29:56.602736 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487fc60e-ad5f-4e86-b1fe-c28f6c69d20b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:29:56.603343 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487fc60e-ad5f-4e86-b1fe-c28f6c69d20b admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:29:56.603653 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487fc60e-ad5f-4e86-b1fe-c28f6c69d20b admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:29:56.603929 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487fc60e-ad5f-4e86-b1fe-c28f6c69d20b admin admin] PATH_INFO: `/v3/domains` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:29:56.604586 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487fc60e-ad5f-4e86-b1fe-c28f6c69d20b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:29:56.604885 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487fc60e-ad5f-4e86-b1fe-c28f6c69d20b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:29:56.607055 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487fc60e-ad5f-4e86-b1fe-c28f6c69d20b admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:29:56.619325 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1049/2105] 10.222.0.16 () {58 vars in 1153 bytes} [Wed Aug 24 09:29:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:29:56.623939 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b0b95d-2c32-4add-be48-dac6b4809a87 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:29:56.633199 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b0b95d-2c32-4add-be48-dac6b4809a87 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:29:56.643678 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c04019f-25ab-4a35-91b1-1b8004d31e28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:29:56.644321 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c04019f-25ab-4a35-91b1-1b8004d31e28 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:29:56.644609 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c04019f-25ab-4a35-91b1-1b8004d31e28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:29:56.645448 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c04019f-25ab-4a35-91b1-1b8004d31e28 admin admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:29:56.646304 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c04019f-25ab-4a35-91b1-1b8004d31e28 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:29:56.648518 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c04019f-25ab-4a35-91b1-1b8004d31e28 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:29:56.768637 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1057/2106] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 24 09:29:56 2022] POST /identity/v3/projects => generated 371 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 09:29:56.770378 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487fc60e-ad5f-4e86-b1fe-c28f6c69d20b admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:29:56.781086 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487fc60e-ad5f-4e86-b1fe-c28f6c69d20b admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:29:56.846791 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034ab579-70ec-4c75-9684-56da6bc32406 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:29:56.847402 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034ab579-70ec-4c75-9684-56da6bc32406 admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:29:56.847709 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034ab579-70ec-4c75-9684-56da6bc32406 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:29:56.847999 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034ab579-70ec-4c75-9684-56da6bc32406 admin admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:29:56.848630 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034ab579-70ec-4c75-9684-56da6bc32406 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:29:56.850537 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034ab579-70ec-4c75-9684-56da6bc32406 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:29:56.861738 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:29:56.979549 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1050/2107] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 24 09:29:56 2022] POST /identity/v3/users => generated 286 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 09:29:56.983989 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c04019f-25ab-4a35-91b1-1b8004d31e28 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:29:56.985955 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e894e2f8-fc5a-4098-8562-95c1db554d97 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:29:56.986243 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e894e2f8-fc5a-4098-8562-95c1db554d97 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:29:56.988060 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e894e2f8-fc5a-4098-8562-95c1db554d97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:29:56.996207 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c04019f-25ab-4a35-91b1-1b8004d31e28 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:29:57.003321 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6015d360-def4-4616-8462-4122499d0629 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:29:57.003321 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6015d360-def4-4616-8462-4122499d0629 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:29:57.003505 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6015d360-def4-4616-8462-4122499d0629 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:29:57.003856 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6015d360-def4-4616-8462-4122499d0629 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:29:57.004546 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6015d360-def4-4616-8462-4122499d0629 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:29:57.007055 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6015d360-def4-4616-8462-4122499d0629 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:29:57.015040 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-e894e2f8-fc5a-4098-8562-95c1db554d97 None None] MFA Rules not processed for user `0aff36cf71984238b8a9b998043ec7be`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:29:57.016425 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-e894e2f8-fc5a-4098-8562-95c1db554d97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:29:57.026178 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1058/2108] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 24 09:29:56 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:29:57.033374 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6015d360-def4-4616-8462-4122499d0629 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:29:57.045853 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6015d360-def4-4616-8462-4122499d0629 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:29:57.054327 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b108323-732c-422a-9b75-3e5d7d4a9ae9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:29:57.054970 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b108323-732c-422a-9b75-3e5d7d4a9ae9 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:29:57.057150 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b108323-732c-422a-9b75-3e5d7d4a9ae9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:29:57.057150 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b108323-732c-422a-9b75-3e5d7d4a9ae9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:29:57.057150 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b108323-732c-422a-9b75-3e5d7d4a9ae9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:29:57.058240 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b108323-732c-422a-9b75-3e5d7d4a9ae9 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:29:57.081690 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1059/2109] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 24 09:29:57 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 09:29:57.085460 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b108323-732c-422a-9b75-3e5d7d4a9ae9 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:29:57.094213 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1051/2110] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 09:29:56 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:29:57.099808 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b108323-732c-422a-9b75-3e5d7d4a9ae9 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:29:57.104893 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b5d3da-3bb9-47bf-8daa-32682d1e8578 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:29:57.108743 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b5d3da-3bb9-47bf-8daa-32682d1e8578 admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:29:57.108743 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b5d3da-3bb9-47bf-8daa-32682d1e8578 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:29:57.108743 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b5d3da-3bb9-47bf-8daa-32682d1e8578 admin admin] PATH_INFO: `/v3/projects/1e0c2d44897d415cb0f7618fb128b3cb/users/89e93708d4d44c40a0558b4106034eb8/roles/136cdf78e00a426997dd4e37faff489b` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:29:57.123208 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b5d3da-3bb9-47bf-8daa-32682d1e8578 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e0c2d44897d415cb0f7618fb128b3cb, user_id=89e93708d4d44c40a0558b4106034eb8, role_id=136cdf78e00a426997dd4e37faff489b)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:29:57.127158 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b5d3da-3bb9-47bf-8daa-32682d1e8578 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:29:57.209238 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1060/2111] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 24 09:29:57 2022] PUT /identity/v3/projects/1e0c2d44897d415cb0f7618fb128b3cb/users/89e93708d4d44c40a0558b4106034eb8/roles/136cdf78e00a426997dd4e37faff489b => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:29:57.221287 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e098bc-7162-45e5-b081-e75cbf5e5081 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:29:57.221577 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e098bc-7162-45e5-b081-e75cbf5e5081 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:29:57.221852 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e098bc-7162-45e5-b081-e75cbf5e5081 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:29:57.255716 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-e7e098bc-7162-45e5-b081-e75cbf5e5081 None None] MFA Rules not processed for user `89e93708d4d44c40a0558b4106034eb8`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:29:57.256992 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-e7e098bc-7162-45e5-b081-e75cbf5e5081 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:29:57.340947 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1052/2112] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 09:29:57 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:29:57.352500 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b5d3da-3bb9-47bf-8daa-32682d1e8578 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:29:57.354728 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-01b5d3da-3bb9-47bf-8daa-32682d1e8578 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:29:57.455737 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b5d3da-3bb9-47bf-8daa-32682d1e8578 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:29:57.460006 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aff0756-a73f-4f21-9bf4-95e68a79a69c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c32b83577c46488906fe409a790b24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:29:57.460620 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aff0756-a73f-4f21-9bf4-95e68a79a69c service cinder] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:29:57.460901 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aff0756-a73f-4f21-9bf4-95e68a79a69c service cinder] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:29:57.461196 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aff0756-a73f-4f21-9bf4-95e68a79a69c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:29:57.483685 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aff0756-a73f-4f21-9bf4-95e68a79a69c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:29:57.483685 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aff0756-a73f-4f21-9bf4-95e68a79a69c service cinder] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:29:57.503696 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1061/2113] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:29:57 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:29:57.551814 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e098bc-7162-45e5-b081-e75cbf5e5081 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:29:57.677056 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e098bc-7162-45e5-b081-e75cbf5e5081 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:29:57.683682 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cdebc0-e7cf-4fc4-80bc-e9dbc5b588e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:29:57.684570 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cdebc0-e7cf-4fc4-80bc-e9dbc5b588e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:29:57.684857 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cdebc0-e7cf-4fc4-80bc-e9dbc5b588e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:29:57.685999 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cdebc0-e7cf-4fc4-80bc-e9dbc5b588e6 admin admin] PATH_INFO: `/v3/users/89e93708d4d44c40a0558b4106034eb8` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:29:57.688745 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cdebc0-e7cf-4fc4-80bc-e9dbc5b588e6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=89e93708d4d44c40a0558b4106034eb8)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:29:57.691665 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cdebc0-e7cf-4fc4-80bc-e9dbc5b588e6 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:29:57.902128 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:29:57.920487 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-74cdebc0-e7cf-4fc4-80bc-e9dbc5b588e6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '89e93708d4d44c40a0558b4106034eb8'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:29:57.921309 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-74cdebc0-e7cf-4fc4-80bc-e9dbc5b588e6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '89e93708d4d44c40a0558b4106034eb8'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:29:57.936537 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-74cdebc0-e7cf-4fc4-80bc-e9dbc5b588e6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '89e93708d4d44c40a0558b4106034eb8'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:29:57.943931 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-74cdebc0-e7cf-4fc4-80bc-e9dbc5b588e6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '89e93708d4d44c40a0558b4106034eb8'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:29:58.077598 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-74cdebc0-e7cf-4fc4-80bc-e9dbc5b588e6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '89e93708d4d44c40a0558b4106034eb8'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:29:58.085260 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1053/2114] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 24 09:29:57 2022] DELETE /identity/v3/users/89e93708d4d44c40a0558b4106034eb8 => generated 0 bytes in 534 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:29:58.090390 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb6d012-f025-4d4a-bc6d-7446c8551fd2 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:29:58.090875 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb6d012-f025-4d4a-bc6d-7446c8551fd2 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:29:58.090985 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb6d012-f025-4d4a-bc6d-7446c8551fd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:29:58.112760 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-7eb6d012-f025-4d4a-bc6d-7446c8551fd2 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:29:58.113967 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-7eb6d012-f025-4d4a-bc6d-7446c8551fd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:29:58.171289 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:29:58.180878 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1062/2115] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 09:29:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:29:58.191220 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cdebc0-e7cf-4fc4-80bc-e9dbc5b588e6 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:29:58.192756 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-74cdebc0-e7cf-4fc4-80bc-e9dbc5b588e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:29:58.254763 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cdebc0-e7cf-4fc4-80bc-e9dbc5b588e6 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:29:58.258511 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86418c01-f41a-4b3a-94be-2961d7de8b24 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e5260e798404e8bb12b25ca8ea97052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:29:58.259138 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86418c01-f41a-4b3a-94be-2961d7de8b24 service neutron] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:29:58.259420 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86418c01-f41a-4b3a-94be-2961d7de8b24 service neutron] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:29:58.259728 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86418c01-f41a-4b3a-94be-2961d7de8b24 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:29:58.268905 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86418c01-f41a-4b3a-94be-2961d7de8b24 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:29:58.270398 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86418c01-f41a-4b3a-94be-2961d7de8b24 service neutron] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:29:58.280146 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1054/2116] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:29:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:29:58.736688 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb6d012-f025-4d4a-bc6d-7446c8551fd2 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:29:58.738331 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-7eb6d012-f025-4d4a-bc6d-7446c8551fd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:29:58.760471 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb6d012-f025-4d4a-bc6d-7446c8551fd2 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:29:58.799626 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbc8ff9-d2dd-4701-af78-86c66dd2ffc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:29:58.800406 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbc8ff9-d2dd-4701-af78-86c66dd2ffc5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:29:58.800752 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbc8ff9-d2dd-4701-af78-86c66dd2ffc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:29:58.802727 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbc8ff9-d2dd-4701-af78-86c66dd2ffc5 admin admin] PATH_INFO: `/v3/projects/1e0c2d44897d415cb0f7618fb128b3cb` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:29:58.804450 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbc8ff9-d2dd-4701-af78-86c66dd2ffc5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e0c2d44897d415cb0f7618fb128b3cb)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:29:58.808821 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbc8ff9-d2dd-4701-af78-86c66dd2ffc5 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:29:58.991982 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-cbbc8ff9-d2dd-4701-af78-86c66dd2ffc5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e0c2d44897d415cb0f7618fb128b3cb', 'request_id': 'req-cbbc8ff9-d2dd-4701-af78-86c66dd2ffc5'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:29:59.000280 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-cbbc8ff9-d2dd-4701-af78-86c66dd2ffc5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e0c2d44897d415cb0f7618fb128b3cb', 'request_id': 'req-cbbc8ff9-d2dd-4701-af78-86c66dd2ffc5'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:29:59.008182 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-cbbc8ff9-d2dd-4701-af78-86c66dd2ffc5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e0c2d44897d415cb0f7618fb128b3cb', 'request_id': 'req-cbbc8ff9-d2dd-4701-af78-86c66dd2ffc5'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:29:59.090796 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-cbbc8ff9-d2dd-4701-af78-86c66dd2ffc5 admin admin] The token cache is being invalidate because project 1e0c2d44897d415cb0f7618fb128b3cb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81602) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:29:59.091380 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-cbbc8ff9-d2dd-4701-af78-86c66dd2ffc5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:29:59.092475 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-cbbc8ff9-d2dd-4701-af78-86c66dd2ffc5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:29:59.094768 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1063/2117] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 24 09:29:58 2022] DELETE /identity/v3/projects/1e0c2d44897d415cb0f7618fb128b3cb => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:30:08.275604 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=43,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:30:08.967517 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86418c01-f41a-4b3a-94be-2961d7de8b24 service neutron] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:30:08.969003 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-86418c01-f41a-4b3a-94be-2961d7de8b24 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:30:08.982204 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86418c01-f41a-4b3a-94be-2961d7de8b24 service neutron] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:30:09.045166 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1997bce0-af43-4996-a6d7-4ccad8bf27b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:30:09.045860 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1997bce0-af43-4996-a6d7-4ccad8bf27b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:30:09.046135 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1997bce0-af43-4996-a6d7-4ccad8bf27b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:30:09.046404 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1997bce0-af43-4996-a6d7-4ccad8bf27b3 admin admin] PATH_INFO: `/v3/users/0b87ff85d31340ddb0dead3d856a40d9` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:30:09.047641 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1997bce0-af43-4996-a6d7-4ccad8bf27b3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b87ff85d31340ddb0dead3d856a40d9)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:30:09.049567 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1997bce0-af43-4996-a6d7-4ccad8bf27b3 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:30:09.085996 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=28,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:30:09.307789 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-1997bce0-af43-4996-a6d7-4ccad8bf27b3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b87ff85d31340ddb0dead3d856a40d9'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:30:09.309105 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-1997bce0-af43-4996-a6d7-4ccad8bf27b3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b87ff85d31340ddb0dead3d856a40d9'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:30:09.320970 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-1997bce0-af43-4996-a6d7-4ccad8bf27b3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b87ff85d31340ddb0dead3d856a40d9'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:30:09.328024 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-1997bce0-af43-4996-a6d7-4ccad8bf27b3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b87ff85d31340ddb0dead3d856a40d9'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:30:09.425117 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-1997bce0-af43-4996-a6d7-4ccad8bf27b3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b87ff85d31340ddb0dead3d856a40d9'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:30:09.431299 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1055/2118] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 24 09:30:08 2022] DELETE /identity/v3/users/0b87ff85d31340ddb0dead3d856a40d9 => generated 0 bytes in 464 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:30:09.436528 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f681f84e-6bb1-4ae6-a076-533613405c31 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:30:09.436818 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f681f84e-6bb1-4ae6-a076-533613405c31 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:30:09.437100 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f681f84e-6bb1-4ae6-a076-533613405c31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:30:09.455782 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-f681f84e-6bb1-4ae6-a076-533613405c31 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:30:09.456909 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-f681f84e-6bb1-4ae6-a076-533613405c31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:30:09.517835 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1064/2119] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 09:30:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:30:09.528007 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1997bce0-af43-4996-a6d7-4ccad8bf27b3 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:30:09.529446 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-1997bce0-af43-4996-a6d7-4ccad8bf27b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:30:09.588215 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1997bce0-af43-4996-a6d7-4ccad8bf27b3 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:30:09.592741 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218726a4-539d-4608-88b2-2709da40b03a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e5260e798404e8bb12b25ca8ea97052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:30:09.593357 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218726a4-539d-4608-88b2-2709da40b03a service neutron] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:30:09.593637 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218726a4-539d-4608-88b2-2709da40b03a service neutron] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:30:09.593914 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218726a4-539d-4608-88b2-2709da40b03a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:30:09.603389 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218726a4-539d-4608-88b2-2709da40b03a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:30:09.605664 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218726a4-539d-4608-88b2-2709da40b03a service neutron] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:30:09.618041 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1056/2120] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:30:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:30:10.003757 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f681f84e-6bb1-4ae6-a076-533613405c31 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:30:10.005274 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-f681f84e-6bb1-4ae6-a076-533613405c31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:30:10.018217 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f681f84e-6bb1-4ae6-a076-533613405c31 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:30:10.023607 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9806e637-e167-49a2-b3db-a34f7684e38c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:30:10.024263 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9806e637-e167-49a2-b3db-a34f7684e38c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:30:10.024542 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9806e637-e167-49a2-b3db-a34f7684e38c admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:30:10.024815 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9806e637-e167-49a2-b3db-a34f7684e38c admin admin] PATH_INFO: `/v3/projects/71b1f6e3a00046bca89ba0e91cdf6c5f` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:30:10.025945 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9806e637-e167-49a2-b3db-a34f7684e38c admin admin] RBAC: Authorizing `identity:delete_project(project_id=71b1f6e3a00046bca89ba0e91cdf6c5f)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:30:10.027980 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9806e637-e167-49a2-b3db-a34f7684e38c admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:30:10.129143 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-9806e637-e167-49a2-b3db-a34f7684e38c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '71b1f6e3a00046bca89ba0e91cdf6c5f', 'request_id': 'req-9806e637-e167-49a2-b3db-a34f7684e38c'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:30:10.135415 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-9806e637-e167-49a2-b3db-a34f7684e38c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '71b1f6e3a00046bca89ba0e91cdf6c5f', 'request_id': 'req-9806e637-e167-49a2-b3db-a34f7684e38c'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:30:10.141586 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-9806e637-e167-49a2-b3db-a34f7684e38c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '71b1f6e3a00046bca89ba0e91cdf6c5f', 'request_id': 'req-9806e637-e167-49a2-b3db-a34f7684e38c'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:30:10.219638 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-9806e637-e167-49a2-b3db-a34f7684e38c admin admin] The token cache is being invalidate because project 71b1f6e3a00046bca89ba0e91cdf6c5f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81602) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:30:10.219935 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-9806e637-e167-49a2-b3db-a34f7684e38c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:30:10.220595 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-9806e637-e167-49a2-b3db-a34f7684e38c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:30:10.222199 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1065/2121] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 24 09:30:10 2022] DELETE /identity/v3/projects/71b1f6e3a00046bca89ba0e91cdf6c5f => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:30:10.227220 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722769cb-040d-4fa0-9734-4015166cd608 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:30:10.227515 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722769cb-040d-4fa0-9734-4015166cd608 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:30:10.227825 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722769cb-040d-4fa0-9734-4015166cd608 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:30:10.248778 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-722769cb-040d-4fa0-9734-4015166cd608 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:30:10.250008 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-722769cb-040d-4fa0-9734-4015166cd608 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:30:10.267493 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:30:10.310570 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1057/2122] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 09:30:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:30:10.318400 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b51f90-244a-48c6-9d92-104480d323d8 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:30:10.318681 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b51f90-244a-48c6-9d92-104480d323d8 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:30:10.318951 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b51f90-244a-48c6-9d92-104480d323d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:30:10.340549 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-a0b51f90-244a-48c6-9d92-104480d323d8 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:30:10.341740 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-a0b51f90-244a-48c6-9d92-104480d323d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:30:10.349496 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1066/2123] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 09:30:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:30:10.353717 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722769cb-040d-4fa0-9734-4015166cd608 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:30:10.363293 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722769cb-040d-4fa0-9734-4015166cd608 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:30:10.367900 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d74bee1-2ab7-4358-a931-e0f754ad665d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:30:10.368500 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d74bee1-2ab7-4358-a931-e0f754ad665d admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:30:10.368782 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d74bee1-2ab7-4358-a931-e0f754ad665d admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:30:10.369049 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d74bee1-2ab7-4358-a931-e0f754ad665d admin admin] PATH_INFO: `/v3/domains` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:30:10.369688 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d74bee1-2ab7-4358-a931-e0f754ad665d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:30:10.369990 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d74bee1-2ab7-4358-a931-e0f754ad665d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:30:10.372132 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d74bee1-2ab7-4358-a931-e0f754ad665d admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:30:10.383507 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1058/2124] 10.222.0.16 () {58 vars in 1153 bytes} [Wed Aug 24 09:30:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:30:10.387824 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b51f90-244a-48c6-9d92-104480d323d8 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:30:10.396829 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b51f90-244a-48c6-9d92-104480d323d8 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:30:10.401570 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0c3c3b-3acb-4736-b3cd-fe99ba211b58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:30:10.402166 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0c3c3b-3acb-4736-b3cd-fe99ba211b58 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:30:10.402443 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0c3c3b-3acb-4736-b3cd-fe99ba211b58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:30:10.402712 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0c3c3b-3acb-4736-b3cd-fe99ba211b58 admin admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:30:10.403338 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0c3c3b-3acb-4736-b3cd-fe99ba211b58 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:30:10.405265 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0c3c3b-3acb-4736-b3cd-fe99ba211b58 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:30:10.471538 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1067/2125] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 24 09:30:10 2022] POST /identity/v3/projects => generated 383 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 09:30:10.475760 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d74bee1-2ab7-4358-a931-e0f754ad665d admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:30:10.485793 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d74bee1-2ab7-4358-a931-e0f754ad665d admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:30:10.554382 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2caba7a5-f227-4ee6-9a56-1a67ad5fb014 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:30:10.554905 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caba7a5-f227-4ee6-9a56-1a67ad5fb014 admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:30:10.555181 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caba7a5-f227-4ee6-9a56-1a67ad5fb014 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:30:10.555451 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caba7a5-f227-4ee6-9a56-1a67ad5fb014 admin admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:30:10.556310 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2caba7a5-f227-4ee6-9a56-1a67ad5fb014 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:30:10.558653 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2caba7a5-f227-4ee6-9a56-1a67ad5fb014 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:30:10.633056 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1059/2126] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 24 09:30:10 2022] POST /identity/v3/users => generated 292 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 09:30:10.638083 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0c3c3b-3acb-4736-b3cd-fe99ba211b58 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:30:10.652240 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0c3c3b-3acb-4736-b3cd-fe99ba211b58 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:30:10.660681 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c50f4d-317d-455b-bbd9-53c9d36f47ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:30:10.661535 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c50f4d-317d-455b-bbd9-53c9d36f47ff admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:30:10.661999 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c50f4d-317d-455b-bbd9-53c9d36f47ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:30:10.662425 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c50f4d-317d-455b-bbd9-53c9d36f47ff admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:30:10.663616 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c50f4d-317d-455b-bbd9-53c9d36f47ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:30:10.665234 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c50f4d-317d-455b-bbd9-53c9d36f47ff admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:30:10.677823 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1068/2127] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 24 09:30:10 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:30:10.681919 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2caba7a5-f227-4ee6-9a56-1a67ad5fb014 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:30:10.695423 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2caba7a5-f227-4ee6-9a56-1a67ad5fb014 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:30:10.703504 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910212d9-b5ad-47d7-87f3-73981c4df8ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:30:10.704168 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910212d9-b5ad-47d7-87f3-73981c4df8ee admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:30:10.704446 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910212d9-b5ad-47d7-87f3-73981c4df8ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:30:10.704713 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910212d9-b5ad-47d7-87f3-73981c4df8ee admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:30:10.705360 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910212d9-b5ad-47d7-87f3-73981c4df8ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:30:10.706938 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910212d9-b5ad-47d7-87f3-73981c4df8ee admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:30:10.719266 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1060/2128] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 24 09:30:10 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:30:10.723507 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c50f4d-317d-455b-bbd9-53c9d36f47ff admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:30:10.734880 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c50f4d-317d-455b-bbd9-53c9d36f47ff admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:30:10.740636 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bcc734-d792-4836-8e8b-43497b02dbdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:30:10.741302 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bcc734-d792-4836-8e8b-43497b02dbdf admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:30:10.741582 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bcc734-d792-4836-8e8b-43497b02dbdf admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:30:10.741873 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bcc734-d792-4836-8e8b-43497b02dbdf admin admin] PATH_INFO: `/v3/projects/ea60b2008af64db383c7cf842292df77/users/6ffb9c3743464fd2a9bc0dc69c29eb94/roles/136cdf78e00a426997dd4e37faff489b` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:30:10.761711 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50bcc734-d792-4836-8e8b-43497b02dbdf admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea60b2008af64db383c7cf842292df77, user_id=6ffb9c3743464fd2a9bc0dc69c29eb94, role_id=136cdf78e00a426997dd4e37faff489b)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:30:10.764719 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50bcc734-d792-4836-8e8b-43497b02dbdf admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:30:10.867446 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1069/2129] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 24 09:30:10 2022] PUT /identity/v3/projects/ea60b2008af64db383c7cf842292df77/users/6ffb9c3743464fd2a9bc0dc69c29eb94/roles/136cdf78e00a426997dd4e37faff489b => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:30:10.878520 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2ec45f-efc5-4527-aa8e-0726e98405fe None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:30:10.878814 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2ec45f-efc5-4527-aa8e-0726e98405fe None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:30:10.879113 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2ec45f-efc5-4527-aa8e-0726e98405fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:30:10.912122 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-3a2ec45f-efc5-4527-aa8e-0726e98405fe None None] MFA Rules not processed for user `6ffb9c3743464fd2a9bc0dc69c29eb94`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:30:10.913449 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-3a2ec45f-efc5-4527-aa8e-0726e98405fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:30:10.978434 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1061/2130] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 09:30:10 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:30:10.989735 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bcc734-d792-4836-8e8b-43497b02dbdf admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:30:10.991511 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-50bcc734-d792-4836-8e8b-43497b02dbdf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:30:11.013883 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:30:11.050347 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bcc734-d792-4836-8e8b-43497b02dbdf admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:30:11.054267 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0343bdd7-5c6b-4297-ad00-c84f9f8d973a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c32b83577c46488906fe409a790b24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:30:11.054878 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0343bdd7-5c6b-4297-ad00-c84f9f8d973a service cinder] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:30:11.055154 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0343bdd7-5c6b-4297-ad00-c84f9f8d973a service cinder] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:30:11.055421 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0343bdd7-5c6b-4297-ad00-c84f9f8d973a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:30:11.065120 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0343bdd7-5c6b-4297-ad00-c84f9f8d973a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:30:11.066553 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0343bdd7-5c6b-4297-ad00-c84f9f8d973a service cinder] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:30:11.077903 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1070/2131] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:30:10 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:30:20.975737 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=89,keystone:INSERT=3 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:30:21.104940 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=17 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:30:29.154554 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2ec45f-efc5-4527-aa8e-0726e98405fe None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:30:29.219008 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2ec45f-efc5-4527-aa8e-0726e98405fe None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:30:29.224273 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b3470c-b66f-4279-83be-5e9a82bf3ae7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:30:29.225015 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b3470c-b66f-4279-83be-5e9a82bf3ae7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:30:29.225766 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b3470c-b66f-4279-83be-5e9a82bf3ae7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:30:29.225766 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b3470c-b66f-4279-83be-5e9a82bf3ae7 admin admin] PATH_INFO: `/v3/users/6ffb9c3743464fd2a9bc0dc69c29eb94` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:30:29.226760 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b3470c-b66f-4279-83be-5e9a82bf3ae7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6ffb9c3743464fd2a9bc0dc69c29eb94)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:30:29.228772 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b3470c-b66f-4279-83be-5e9a82bf3ae7 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:30:29.398700 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-a9b3470c-b66f-4279-83be-5e9a82bf3ae7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6ffb9c3743464fd2a9bc0dc69c29eb94'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:30:29.399294 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-a9b3470c-b66f-4279-83be-5e9a82bf3ae7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6ffb9c3743464fd2a9bc0dc69c29eb94'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:30:29.411204 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-a9b3470c-b66f-4279-83be-5e9a82bf3ae7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6ffb9c3743464fd2a9bc0dc69c29eb94'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:30:29.417961 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-a9b3470c-b66f-4279-83be-5e9a82bf3ae7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6ffb9c3743464fd2a9bc0dc69c29eb94'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:30:29.510549 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-a9b3470c-b66f-4279-83be-5e9a82bf3ae7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6ffb9c3743464fd2a9bc0dc69c29eb94'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:30:29.517169 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1062/2132] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 24 09:30:29 2022] DELETE /identity/v3/users/6ffb9c3743464fd2a9bc0dc69c29eb94 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:30:29.521710 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391b1a8d-f98d-48a0-9055-eeaad5cbbfdf None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:30:29.522229 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391b1a8d-f98d-48a0-9055-eeaad5cbbfdf None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:30:29.522324 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391b1a8d-f98d-48a0-9055-eeaad5cbbfdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:30:29.546703 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-391b1a8d-f98d-48a0-9055-eeaad5cbbfdf None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:30:29.548570 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-391b1a8d-f98d-48a0-9055-eeaad5cbbfdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:30:29.612927 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1071/2133] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 09:30:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:30:29.623278 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b3470c-b66f-4279-83be-5e9a82bf3ae7 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:30:29.624869 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-a9b3470c-b66f-4279-83be-5e9a82bf3ae7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:30:29.684124 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b3470c-b66f-4279-83be-5e9a82bf3ae7 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:30:29.688879 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c08676-0d2b-4fdb-a7f1-6863ccb7bad3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e5260e798404e8bb12b25ca8ea97052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:30:29.689582 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c08676-0d2b-4fdb-a7f1-6863ccb7bad3 service neutron] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:30:29.689867 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c08676-0d2b-4fdb-a7f1-6863ccb7bad3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:30:29.690144 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c08676-0d2b-4fdb-a7f1-6863ccb7bad3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:30:29.702639 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c08676-0d2b-4fdb-a7f1-6863ccb7bad3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:30:29.703575 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c08676-0d2b-4fdb-a7f1-6863ccb7bad3 service neutron] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:30:29.714226 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1063/2134] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:30:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:30:30.168614 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391b1a8d-f98d-48a0-9055-eeaad5cbbfdf None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:30:30.170555 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-391b1a8d-f98d-48a0-9055-eeaad5cbbfdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:30:30.183433 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391b1a8d-f98d-48a0-9055-eeaad5cbbfdf None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:30:30.188586 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfd32d2-dbcc-4563-b41c-97b03f23eadd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:30:30.189230 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfd32d2-dbcc-4563-b41c-97b03f23eadd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:30:30.189509 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfd32d2-dbcc-4563-b41c-97b03f23eadd admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:30:30.189778 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfd32d2-dbcc-4563-b41c-97b03f23eadd admin admin] PATH_INFO: `/v3/projects/ea60b2008af64db383c7cf842292df77` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:30:30.190991 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfd32d2-dbcc-4563-b41c-97b03f23eadd admin admin] RBAC: Authorizing `identity:delete_project(project_id=ea60b2008af64db383c7cf842292df77)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:30:30.200865 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfd32d2-dbcc-4563-b41c-97b03f23eadd admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:30:30.350925 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-cbfd32d2-dbcc-4563-b41c-97b03f23eadd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ea60b2008af64db383c7cf842292df77', 'request_id': 'req-cbfd32d2-dbcc-4563-b41c-97b03f23eadd'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:30:30.371092 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-cbfd32d2-dbcc-4563-b41c-97b03f23eadd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ea60b2008af64db383c7cf842292df77', 'request_id': 'req-cbfd32d2-dbcc-4563-b41c-97b03f23eadd'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:30:30.390400 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-cbfd32d2-dbcc-4563-b41c-97b03f23eadd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ea60b2008af64db383c7cf842292df77', 'request_id': 'req-cbfd32d2-dbcc-4563-b41c-97b03f23eadd'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:30:30.487745 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-cbfd32d2-dbcc-4563-b41c-97b03f23eadd admin admin] The token cache is being invalidate because project ea60b2008af64db383c7cf842292df77 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81602) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:30:30.488040 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-cbfd32d2-dbcc-4563-b41c-97b03f23eadd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:30:30.488795 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-cbfd32d2-dbcc-4563-b41c-97b03f23eadd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:30:30.490600 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1072/2135] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 24 09:30:30 2022] DELETE /identity/v3/projects/ea60b2008af64db383c7cf842292df77 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:30:30.495259 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609be54d-ec35-4678-ae8d-bbe1c6c59adc None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:30:30.495772 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609be54d-ec35-4678-ae8d-bbe1c6c59adc None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:30:30.495867 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609be54d-ec35-4678-ae8d-bbe1c6c59adc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:30:30.515684 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-609be54d-ec35-4678-ae8d-bbe1c6c59adc None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:30:30.516540 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-609be54d-ec35-4678-ae8d-bbe1c6c59adc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:30:30.541445 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:30:30.627797 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1064/2136] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 09:30:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:30:30.636654 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cec9fa-fcd9-4283-abb9-90cba0e8e09e None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:30:30.637133 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cec9fa-fcd9-4283-abb9-90cba0e8e09e None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:30:30.637221 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cec9fa-fcd9-4283-abb9-90cba0e8e09e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:30:30.664319 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-52cec9fa-fcd9-4283-abb9-90cba0e8e09e None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:30:30.665529 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-52cec9fa-fcd9-4283-abb9-90cba0e8e09e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:30:30.677470 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1073/2137] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 09:30:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:30:30.692254 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609be54d-ec35-4678-ae8d-bbe1c6c59adc None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:30:30.692254 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609be54d-ec35-4678-ae8d-bbe1c6c59adc None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:30:30.702158 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a9fbd3-b05e-483e-b092-288b5034ee9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:30:30.702882 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a9fbd3-b05e-483e-b092-288b5034ee9d admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:30:30.703298 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a9fbd3-b05e-483e-b092-288b5034ee9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:30:30.703715 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a9fbd3-b05e-483e-b092-288b5034ee9d admin admin] PATH_INFO: `/v3/domains` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:30:30.704693 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a9fbd3-b05e-483e-b092-288b5034ee9d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:30:30.705119 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a9fbd3-b05e-483e-b092-288b5034ee9d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:30:30.707006 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a9fbd3-b05e-483e-b092-288b5034ee9d admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:30:30.724163 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1065/2138] 10.222.0.16 () {58 vars in 1153 bytes} [Wed Aug 24 09:30:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:30:30.725738 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cec9fa-fcd9-4283-abb9-90cba0e8e09e None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:30:30.735274 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cec9fa-fcd9-4283-abb9-90cba0e8e09e None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:30:30.740586 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f2deb5-f675-46a6-8ff1-58c1a8fad040 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:30:30.741258 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f2deb5-f675-46a6-8ff1-58c1a8fad040 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:30:30.742062 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f2deb5-f675-46a6-8ff1-58c1a8fad040 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:30:30.742062 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f2deb5-f675-46a6-8ff1-58c1a8fad040 admin admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:30:30.742666 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f2deb5-f675-46a6-8ff1-58c1a8fad040 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:30:30.744647 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f2deb5-f675-46a6-8ff1-58c1a8fad040 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:30:30.833852 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1074/2139] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 24 09:30:30 2022] POST /identity/v3/projects => generated 383 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:30:30.838616 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a9fbd3-b05e-483e-b092-288b5034ee9d admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:30:30.851318 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a9fbd3-b05e-483e-b092-288b5034ee9d admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:30:30.909438 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d399cb4e-7359-45a5-b43e-a5bf43e027c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:30:30.910040 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d399cb4e-7359-45a5-b43e-a5bf43e027c6 admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:30:30.910323 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d399cb4e-7359-45a5-b43e-a5bf43e027c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:30:30.910598 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d399cb4e-7359-45a5-b43e-a5bf43e027c6 admin admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:30:30.911216 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d399cb4e-7359-45a5-b43e-a5bf43e027c6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:30:30.913169 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d399cb4e-7359-45a5-b43e-a5bf43e027c6 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:30:31.016556 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1066/2140] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 24 09:30:30 2022] POST /identity/v3/users => generated 292 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:30:31.017745 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f2deb5-f675-46a6-8ff1-58c1a8fad040 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:30:31.028388 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f2deb5-f675-46a6-8ff1-58c1a8fad040 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:30:31.033021 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b506c1e8-b5bf-4075-aa56-b1923c66f944 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:30:31.033618 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b506c1e8-b5bf-4075-aa56-b1923c66f944 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:30:31.033897 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b506c1e8-b5bf-4075-aa56-b1923c66f944 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:30:31.034167 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b506c1e8-b5bf-4075-aa56-b1923c66f944 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:30:31.034803 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b506c1e8-b5bf-4075-aa56-b1923c66f944 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:30:31.036571 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b506c1e8-b5bf-4075-aa56-b1923c66f944 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:30:31.047241 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1075/2141] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 24 09:30:31 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:30:31.051177 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d399cb4e-7359-45a5-b43e-a5bf43e027c6 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:30:31.062001 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d399cb4e-7359-45a5-b43e-a5bf43e027c6 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:30:31.066878 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e87461b-72e8-4efc-abaf-186ac16afa79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:30:31.067469 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e87461b-72e8-4efc-abaf-186ac16afa79 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:30:31.067776 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e87461b-72e8-4efc-abaf-186ac16afa79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:30:31.068051 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e87461b-72e8-4efc-abaf-186ac16afa79 admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:30:31.068809 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e87461b-72e8-4efc-abaf-186ac16afa79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:30:31.070420 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e87461b-72e8-4efc-abaf-186ac16afa79 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:30:31.081970 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1067/2142] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 24 09:30:31 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:30:31.086074 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b506c1e8-b5bf-4075-aa56-b1923c66f944 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:30:31.096606 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b506c1e8-b5bf-4075-aa56-b1923c66f944 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:30:31.101519 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809811d5-c4a9-4dc6-b044-f8ec42456c9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:30:31.102208 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809811d5-c4a9-4dc6-b044-f8ec42456c9d admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:30:31.102492 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809811d5-c4a9-4dc6-b044-f8ec42456c9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:30:31.102767 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809811d5-c4a9-4dc6-b044-f8ec42456c9d admin admin] PATH_INFO: `/v3/projects/0b3e1d0646174986919db290c6da2579/users/bfc3f904a4fc4abc91e8df52ec816a0a/roles/136cdf78e00a426997dd4e37faff489b` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:30:31.119210 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809811d5-c4a9-4dc6-b044-f8ec42456c9d admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b3e1d0646174986919db290c6da2579, user_id=bfc3f904a4fc4abc91e8df52ec816a0a, role_id=136cdf78e00a426997dd4e37faff489b)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:30:31.124093 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809811d5-c4a9-4dc6-b044-f8ec42456c9d admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:30:31.225112 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1076/2143] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 24 09:30:31 2022] PUT /identity/v3/projects/0b3e1d0646174986919db290c6da2579/users/bfc3f904a4fc4abc91e8df52ec816a0a/roles/136cdf78e00a426997dd4e37faff489b => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 09:30:31.230199 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562b374b-4594-454b-b33b-7b1f99ec27fd None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:30:31.230481 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562b374b-4594-454b-b33b-7b1f99ec27fd None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:30:31.230751 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562b374b-4594-454b-b33b-7b1f99ec27fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:30:31.250166 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-562b374b-4594-454b-b33b-7b1f99ec27fd None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:30:31.251359 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-562b374b-4594-454b-b33b-7b1f99ec27fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:30:31.318576 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1068/2144] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 09:30:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:30:31.329058 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809811d5-c4a9-4dc6-b044-f8ec42456c9d admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:30:31.330786 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-809811d5-c4a9-4dc6-b044-f8ec42456c9d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:30:31.353941 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:30:31.389614 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809811d5-c4a9-4dc6-b044-f8ec42456c9d admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:30:31.393385 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5c9faa-c74e-4ef6-888a-ac29915db6a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e5260e798404e8bb12b25ca8ea97052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:30:31.394019 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5c9faa-c74e-4ef6-888a-ac29915db6a1 service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:30:31.394328 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5c9faa-c74e-4ef6-888a-ac29915db6a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:30:31.394649 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5c9faa-c74e-4ef6-888a-ac29915db6a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:30:31.404588 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5c9faa-c74e-4ef6-888a-ac29915db6a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:30:31.406041 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5c9faa-c74e-4ef6-888a-ac29915db6a1 service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:30:31.415378 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1077/2145] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:30:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:30:32.574317 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809b69cb-2f2d-4315-bb37-7708de3a380b None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:30:32.574900 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809b69cb-2f2d-4315-bb37-7708de3a380b None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:30:32.574900 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809b69cb-2f2d-4315-bb37-7708de3a380b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:30:32.608870 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-809b69cb-2f2d-4315-bb37-7708de3a380b None None] MFA Rules not processed for user `bfc3f904a4fc4abc91e8df52ec816a0a`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:30:32.610140 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-809b69cb-2f2d-4315-bb37-7708de3a380b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:30:32.627451 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:30:32.691420 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1069/2146] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 09:30:32 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:30:32.699707 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5c9faa-c74e-4ef6-888a-ac29915db6a1 service neutron] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:30:32.701675 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-3e5c9faa-c74e-4ef6-888a-ac29915db6a1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:30:32.777428 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5c9faa-c74e-4ef6-888a-ac29915db6a1 service neutron] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:30:32.786437 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f796a7ec-917e-4637-9e6a-247f4170a477 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c32b83577c46488906fe409a790b24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:30:32.787071 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f796a7ec-917e-4637-9e6a-247f4170a477 service cinder] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:30:32.787347 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f796a7ec-917e-4637-9e6a-247f4170a477 service cinder] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:30:32.788267 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f796a7ec-917e-4637-9e6a-247f4170a477 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:30:32.804534 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f796a7ec-917e-4637-9e6a-247f4170a477 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:30:32.806405 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f796a7ec-917e-4637-9e6a-247f4170a477 service cinder] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:30:32.827628 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1078/2147] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:30:32 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:30:42.218169 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee7fe54-e6d8-4a1e-b6aa-446cc5b91cb5 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:30:42.219108 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee7fe54-e6d8-4a1e-b6aa-446cc5b91cb5 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:30:42.219108 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee7fe54-e6d8-4a1e-b6aa-446cc5b91cb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:30:42.239758 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-4ee7fe54-e6d8-4a1e-b6aa-446cc5b91cb5 None None] MFA Rules not processed for user `0489462287fc4aea8e8e720af24e1ef8`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:30:42.240985 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-4ee7fe54-e6d8-4a1e-b6aa-446cc5b91cb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:30:42.299967 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1070/2148] 10.222.0.16 () {58 vars in 1009 bytes} [Wed Aug 24 09:30:42 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:30:42.349156 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f796a7ec-917e-4637-9e6a-247f4170a477 service cinder] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:30:42.350585 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-f796a7ec-917e-4637-9e6a-247f4170a477 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:30:42.426260 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f796a7ec-917e-4637-9e6a-247f4170a477 service cinder] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:30:42.431623 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef5eb76-da5e-4a7e-850b-08e6db2c6286 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdd5f0d67fe54e9fae6f4af644988d93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:30:42.432276 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef5eb76-da5e-4a7e-850b-08e6db2c6286 service swift] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:30:42.432567 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef5eb76-da5e-4a7e-850b-08e6db2c6286 service swift] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:30:42.432846 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef5eb76-da5e-4a7e-850b-08e6db2c6286 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:30:42.441266 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef5eb76-da5e-4a7e-850b-08e6db2c6286 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:30:42.442897 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef5eb76-da5e-4a7e-850b-08e6db2c6286 service swift] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:30:42.451924 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1079/2149] 10.222.0.16 () {60 vars in 1384 bytes} [Wed Aug 24 09:30:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 09:30:43.139341 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754aa77d-1b02-44c4-9e91-9271f0e4df73 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:30:43.140969 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754aa77d-1b02-44c4-9e91-9271f0e4df73 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:30:43.141520 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754aa77d-1b02-44c4-9e91-9271f0e4df73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:30:43.177123 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-754aa77d-1b02-44c4-9e91-9271f0e4df73 None None] MFA Rules not processed for user `bfc3f904a4fc4abc91e8df52ec816a0a`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:30:43.178751 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-754aa77d-1b02-44c4-9e91-9271f0e4df73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:30:43.187877 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1071/2150] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 09:30:43 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:30:49.765463 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef5eb76-da5e-4a7e-850b-08e6db2c6286 service swift] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:30:49.767114 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-4ef5eb76-da5e-4a7e-850b-08e6db2c6286 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:30:49.783426 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef5eb76-da5e-4a7e-850b-08e6db2c6286 service swift] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:30:49.788358 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac361cc-8757-4e2f-b1fd-80965ba82937 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:30:49.789083 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac361cc-8757-4e2f-b1fd-80965ba82937 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:30:49.789437 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac361cc-8757-4e2f-b1fd-80965ba82937 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:30:49.789831 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac361cc-8757-4e2f-b1fd-80965ba82937 admin admin] PATH_INFO: `/v3/users/a9a36d5a7e4b4178807fd9f0fab2f350` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:30:49.791041 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac361cc-8757-4e2f-b1fd-80965ba82937 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a9a36d5a7e4b4178807fd9f0fab2f350)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:30:49.793137 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac361cc-8757-4e2f-b1fd-80965ba82937 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:30:50.042481 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-5ac361cc-8757-4e2f-b1fd-80965ba82937 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a9a36d5a7e4b4178807fd9f0fab2f350'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:30:50.043377 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-5ac361cc-8757-4e2f-b1fd-80965ba82937 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a9a36d5a7e4b4178807fd9f0fab2f350'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:30:50.046982 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:30:50.179230 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-5ac361cc-8757-4e2f-b1fd-80965ba82937 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a9a36d5a7e4b4178807fd9f0fab2f350'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:30:50.181603 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-5ac361cc-8757-4e2f-b1fd-80965ba82937 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a9a36d5a7e4b4178807fd9f0fab2f350'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:30:50.190197 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-5ac361cc-8757-4e2f-b1fd-80965ba82937 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a9a36d5a7e4b4178807fd9f0fab2f350'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:30:50.218060 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1080/2151] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 24 09:30:49 2022] DELETE /identity/v3/users/a9a36d5a7e4b4178807fd9f0fab2f350 => generated 0 bytes in 450 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:30:50.220229 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9200162f-5d67-4a21-9b2e-2888592d228d None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:30:50.221587 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9200162f-5d67-4a21-9b2e-2888592d228d None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:30:50.221587 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9200162f-5d67-4a21-9b2e-2888592d228d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:30:50.243035 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-9200162f-5d67-4a21-9b2e-2888592d228d None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:30:50.244471 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-9200162f-5d67-4a21-9b2e-2888592d228d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:30:50.339092 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1072/2152] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 09:30:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 09:30:50.351258 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac361cc-8757-4e2f-b1fd-80965ba82937 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:30:50.352967 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-5ac361cc-8757-4e2f-b1fd-80965ba82937 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:30:50.435547 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac361cc-8757-4e2f-b1fd-80965ba82937 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:30:50.440788 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efaf27b4-ed40-497c-b942-456177136e0f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e5260e798404e8bb12b25ca8ea97052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:30:50.441158 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaf27b4-ed40-497c-b942-456177136e0f service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:30:50.441542 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaf27b4-ed40-497c-b942-456177136e0f service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:30:50.441933 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaf27b4-ed40-497c-b942-456177136e0f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:30:50.453334 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efaf27b4-ed40-497c-b942-456177136e0f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:30:50.455973 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efaf27b4-ed40-497c-b942-456177136e0f service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:30:50.467604 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1081/2153] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:30:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:30:50.948074 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9200162f-5d67-4a21-9b2e-2888592d228d None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:30:50.949646 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-9200162f-5d67-4a21-9b2e-2888592d228d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:30:50.975146 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9200162f-5d67-4a21-9b2e-2888592d228d None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:30:50.982388 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d889dd50-faff-417d-87b0-40813041c120 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:30:50.983096 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d889dd50-faff-417d-87b0-40813041c120 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:30:50.983694 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d889dd50-faff-417d-87b0-40813041c120 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:30:50.988528 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d889dd50-faff-417d-87b0-40813041c120 admin admin] PATH_INFO: `/v3/projects/d1d26b18d3d0403797d1fd0d3ccdf85d` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:30:50.989986 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d889dd50-faff-417d-87b0-40813041c120 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d1d26b18d3d0403797d1fd0d3ccdf85d)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:30:50.992110 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d889dd50-faff-417d-87b0-40813041c120 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:30:51.137832 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-d889dd50-faff-417d-87b0-40813041c120 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd1d26b18d3d0403797d1fd0d3ccdf85d', 'request_id': 'req-d889dd50-faff-417d-87b0-40813041c120'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:30:51.140757 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:30:51.145782 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-d889dd50-faff-417d-87b0-40813041c120 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd1d26b18d3d0403797d1fd0d3ccdf85d', 'request_id': 'req-d889dd50-faff-417d-87b0-40813041c120'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:30:51.152813 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-d889dd50-faff-417d-87b0-40813041c120 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd1d26b18d3d0403797d1fd0d3ccdf85d', 'request_id': 'req-d889dd50-faff-417d-87b0-40813041c120'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:30:51.284592 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-d889dd50-faff-417d-87b0-40813041c120 admin admin] The token cache is being invalidate because project d1d26b18d3d0403797d1fd0d3ccdf85d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:30:51.284592 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-d889dd50-faff-417d-87b0-40813041c120 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:30:51.284592 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-d889dd50-faff-417d-87b0-40813041c120 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:30:51.284592 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1073/2154] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 24 09:30:50 2022] DELETE /identity/v3/projects/d1d26b18d3d0403797d1fd0d3ccdf85d => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:30:51.290864 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd39100d-cae3-4753-87e4-a6520527736d None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:30:51.290864 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd39100d-cae3-4753-87e4-a6520527736d None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:30:51.290864 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd39100d-cae3-4753-87e4-a6520527736d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:30:51.343882 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-dd39100d-cae3-4753-87e4-a6520527736d None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:30:51.345870 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-dd39100d-cae3-4753-87e4-a6520527736d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:30:51.473374 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1082/2155] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 09:30:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:30:51.484209 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1b4ecd-e1be-4b72-b3fe-c4d9a786176b None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:30:51.484434 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1b4ecd-e1be-4b72-b3fe-c4d9a786176b None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:30:51.484701 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1b4ecd-e1be-4b72-b3fe-c4d9a786176b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:30:51.506512 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-7e1b4ecd-e1be-4b72-b3fe-c4d9a786176b None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:30:51.507804 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-7e1b4ecd-e1be-4b72-b3fe-c4d9a786176b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:30:51.516823 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1074/2156] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 09:30:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:30:51.520975 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd39100d-cae3-4753-87e4-a6520527736d None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:30:51.534500 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd39100d-cae3-4753-87e4-a6520527736d None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:30:51.541438 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dea94d0-5c4e-4b15-9033-20f61b45b341 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:30:51.542118 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dea94d0-5c4e-4b15-9033-20f61b45b341 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:30:51.542503 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dea94d0-5c4e-4b15-9033-20f61b45b341 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:30:51.542890 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dea94d0-5c4e-4b15-9033-20f61b45b341 admin admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:30:51.543733 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dea94d0-5c4e-4b15-9033-20f61b45b341 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:30:51.544118 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dea94d0-5c4e-4b15-9033-20f61b45b341 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:30:51.546651 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dea94d0-5c4e-4b15-9033-20f61b45b341 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:30:51.561821 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1083/2157] 10.222.0.16 () {58 vars in 1153 bytes} [Wed Aug 24 09:30:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:30:51.565714 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1b4ecd-e1be-4b72-b3fe-c4d9a786176b None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:30:51.576706 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1b4ecd-e1be-4b72-b3fe-c4d9a786176b None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:30:51.581869 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712936d5-3c11-4859-97e7-bdff95d5ebf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:30:51.582857 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712936d5-3c11-4859-97e7-bdff95d5ebf9 admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:30:51.583232 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712936d5-3c11-4859-97e7-bdff95d5ebf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:30:51.583751 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712936d5-3c11-4859-97e7-bdff95d5ebf9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:30:51.584438 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712936d5-3c11-4859-97e7-bdff95d5ebf9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:30:51.586715 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712936d5-3c11-4859-97e7-bdff95d5ebf9 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:30:51.662229 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1075/2158] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 24 09:30:51 2022] POST /identity/v3/projects => generated 363 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:30:51.666520 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dea94d0-5c4e-4b15-9033-20f61b45b341 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:30:51.677826 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dea94d0-5c4e-4b15-9033-20f61b45b341 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:30:51.736270 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:DELETE=3,keystone:SELECT=97 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:30:51.742912 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021083af-6c19-4a82-8c9a-a0baefed9072 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:30:51.743642 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021083af-6c19-4a82-8c9a-a0baefed9072 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:30:51.744014 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021083af-6c19-4a82-8c9a-a0baefed9072 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:30:51.744347 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021083af-6c19-4a82-8c9a-a0baefed9072 admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:30:51.745112 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021083af-6c19-4a82-8c9a-a0baefed9072 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:30:51.747150 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021083af-6c19-4a82-8c9a-a0baefed9072 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:30:51.808520 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1084/2159] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 24 09:30:51 2022] POST /identity/v3/users => generated 282 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:30:51.812741 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712936d5-3c11-4859-97e7-bdff95d5ebf9 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:30:51.827772 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712936d5-3c11-4859-97e7-bdff95d5ebf9 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:30:51.836128 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991eee95-a5ad-4524-ae55-38ed4b33486a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:30:51.837450 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991eee95-a5ad-4524-ae55-38ed4b33486a admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:30:51.837930 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991eee95-a5ad-4524-ae55-38ed4b33486a admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:30:51.838372 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991eee95-a5ad-4524-ae55-38ed4b33486a admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:30:51.839416 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991eee95-a5ad-4524-ae55-38ed4b33486a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:30:51.841797 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991eee95-a5ad-4524-ae55-38ed4b33486a admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:30:51.857817 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1076/2160] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 24 09:30:51 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 09:30:51.861989 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021083af-6c19-4a82-8c9a-a0baefed9072 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:30:51.879754 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021083af-6c19-4a82-8c9a-a0baefed9072 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:30:51.887076 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88130071-a497-49fc-8879-34d50d26f624 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:30:51.888036 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88130071-a497-49fc-8879-34d50d26f624 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:30:51.888364 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88130071-a497-49fc-8879-34d50d26f624 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:30:51.888903 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88130071-a497-49fc-8879-34d50d26f624 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:30:51.889815 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88130071-a497-49fc-8879-34d50d26f624 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:30:51.891982 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88130071-a497-49fc-8879-34d50d26f624 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:30:51.908556 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1085/2161] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 24 09:30:51 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 09:30:51.910213 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991eee95-a5ad-4524-ae55-38ed4b33486a admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:30:51.921740 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991eee95-a5ad-4524-ae55-38ed4b33486a admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:30:51.928619 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1fa66e-48ed-4d15-8260-edb1b96e2b28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:30:51.929594 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1fa66e-48ed-4d15-8260-edb1b96e2b28 admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:30:51.930076 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1fa66e-48ed-4d15-8260-edb1b96e2b28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:30:51.930437 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1fa66e-48ed-4d15-8260-edb1b96e2b28 admin admin] PATH_INFO: `/v3/projects/b6168e9f52e6476dad91def3642ac5b7/users/da3da89b69264fe98f264c861c215186/roles/136cdf78e00a426997dd4e37faff489b` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:30:51.951038 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1fa66e-48ed-4d15-8260-edb1b96e2b28 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6168e9f52e6476dad91def3642ac5b7, user_id=da3da89b69264fe98f264c861c215186, role_id=136cdf78e00a426997dd4e37faff489b)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:30:51.953767 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1fa66e-48ed-4d15-8260-edb1b96e2b28 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:30:52.022171 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1077/2162] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 24 09:30:51 2022] PUT /identity/v3/projects/b6168e9f52e6476dad91def3642ac5b7/users/da3da89b69264fe98f264c861c215186/roles/136cdf78e00a426997dd4e37faff489b => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:30:52.022374 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1383060-e35b-4450-948e-74ae959b30f2 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:30:52.022374 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1383060-e35b-4450-948e-74ae959b30f2 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:30:52.022374 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1383060-e35b-4450-948e-74ae959b30f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:30:52.059032 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-b1383060-e35b-4450-948e-74ae959b30f2 None None] MFA Rules not processed for user `da3da89b69264fe98f264c861c215186`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:30:52.060568 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-b1383060-e35b-4450-948e-74ae959b30f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:30:52.138189 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1086/2163] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 09:30:52 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:30:52.150034 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1fa66e-48ed-4d15-8260-edb1b96e2b28 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:30:52.153038 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-6e1fa66e-48ed-4d15-8260-edb1b96e2b28 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:30:52.225512 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1fa66e-48ed-4d15-8260-edb1b96e2b28 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:30:52.229310 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da8fbeb-298d-4aa3-8be3-bb8627ef368e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c32b83577c46488906fe409a790b24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:30:52.230191 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da8fbeb-298d-4aa3-8be3-bb8627ef368e service cinder] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:30:52.230191 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da8fbeb-298d-4aa3-8be3-bb8627ef368e service cinder] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:30:52.230483 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da8fbeb-298d-4aa3-8be3-bb8627ef368e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:30:52.239354 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da8fbeb-298d-4aa3-8be3-bb8627ef368e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:30:52.240923 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da8fbeb-298d-4aa3-8be3-bb8627ef368e service cinder] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:30:52.251103 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1078/2164] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:30:52 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:31:02.134120 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=40,keystone:INSERT=3 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:31:02.246085 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=58,keystone:DELETE=3,keystone:INSERT=3 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:31:02.464003 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849d1c89-4a17-4c66-b253-d51d0d18b887 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:31:02.464291 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849d1c89-4a17-4c66-b253-d51d0d18b887 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:31:02.464565 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849d1c89-4a17-4c66-b253-d51d0d18b887 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:31:02.483915 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-849d1c89-4a17-4c66-b253-d51d0d18b887 None None] MFA Rules not processed for user `0489462287fc4aea8e8e720af24e1ef8`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:31:02.485109 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-849d1c89-4a17-4c66-b253-d51d0d18b887 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:31:02.540528 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1087/2165] 10.222.0.16 () {58 vars in 967 bytes} [Wed Aug 24 09:31:02 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:31:02.554114 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da8fbeb-298d-4aa3-8be3-bb8627ef368e service cinder] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:31:02.555546 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-4da8fbeb-298d-4aa3-8be3-bb8627ef368e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:31:02.627667 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da8fbeb-298d-4aa3-8be3-bb8627ef368e service cinder] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:31:02.632656 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1239afa7-081d-4ce6-a11c-f2d1df017545 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdd5f0d67fe54e9fae6f4af644988d93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:31:02.633264 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1239afa7-081d-4ce6-a11c-f2d1df017545 service swift] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:31:02.633545 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1239afa7-081d-4ce6-a11c-f2d1df017545 service swift] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:31:02.633822 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1239afa7-081d-4ce6-a11c-f2d1df017545 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:31:02.641629 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1239afa7-081d-4ce6-a11c-f2d1df017545 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:31:02.643416 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1239afa7-081d-4ce6-a11c-f2d1df017545 service swift] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:31:02.651290 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1079/2166] 10.222.0.16 () {60 vars in 1384 bytes} [Wed Aug 24 09:31:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 09:31:05.793698 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849d1c89-4a17-4c66-b253-d51d0d18b887 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:31:05.795358 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-849d1c89-4a17-4c66-b253-d51d0d18b887 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:31:05.868031 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849d1c89-4a17-4c66-b253-d51d0d18b887 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:31:05.874393 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3a44b6-6d07-4f9f-aaea-1eae4ed2f3ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:31:05.885112 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3a44b6-6d07-4f9f-aaea-1eae4ed2f3ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:31:05.885112 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3a44b6-6d07-4f9f-aaea-1eae4ed2f3ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:31:05.885112 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3a44b6-6d07-4f9f-aaea-1eae4ed2f3ee admin admin] PATH_INFO: `/v3/users/b3cd284afb854b1e987e807127752925` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:31:05.885112 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3a44b6-6d07-4f9f-aaea-1eae4ed2f3ee admin admin] RBAC: Authorizing `identity:delete_user(user_id=b3cd284afb854b1e987e807127752925)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:31:05.885112 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3a44b6-6d07-4f9f-aaea-1eae4ed2f3ee admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:31:06.072159 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-bf3a44b6-6d07-4f9f-aaea-1eae4ed2f3ee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b3cd284afb854b1e987e807127752925'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:31:06.072888 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-bf3a44b6-6d07-4f9f-aaea-1eae4ed2f3ee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b3cd284afb854b1e987e807127752925'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:31:06.184377 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-bf3a44b6-6d07-4f9f-aaea-1eae4ed2f3ee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b3cd284afb854b1e987e807127752925'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:31:06.184928 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-bf3a44b6-6d07-4f9f-aaea-1eae4ed2f3ee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b3cd284afb854b1e987e807127752925'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:31:06.193507 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-bf3a44b6-6d07-4f9f-aaea-1eae4ed2f3ee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b3cd284afb854b1e987e807127752925'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:31:06.214988 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1088/2167] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 24 09:31:05 2022] DELETE /identity/v3/users/b3cd284afb854b1e987e807127752925 => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:31:06.489651 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1239afa7-081d-4ce6-a11c-f2d1df017545 service swift] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:31:06.500843 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1239afa7-081d-4ce6-a11c-f2d1df017545 service swift] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:31:06.570311 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a7c77e-d7b1-4a88-af08-a1007094f5bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:31:06.570943 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a7c77e-d7b1-4a88-af08-a1007094f5bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:31:06.571261 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a7c77e-d7b1-4a88-af08-a1007094f5bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:31:06.571537 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a7c77e-d7b1-4a88-af08-a1007094f5bc admin admin] PATH_INFO: `/v3/projects/6204c9e825f841aab905eac66e6a4a1e` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:31:06.572680 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a7c77e-d7b1-4a88-af08-a1007094f5bc admin admin] RBAC: Authorizing `identity:delete_project(project_id=6204c9e825f841aab905eac66e6a4a1e)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:31:06.574591 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a7c77e-d7b1-4a88-af08-a1007094f5bc admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:31:06.710542 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-e3a7c77e-d7b1-4a88-af08-a1007094f5bc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6204c9e825f841aab905eac66e6a4a1e', 'request_id': 'req-e3a7c77e-d7b1-4a88-af08-a1007094f5bc'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:31:06.717299 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-e3a7c77e-d7b1-4a88-af08-a1007094f5bc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6204c9e825f841aab905eac66e6a4a1e', 'request_id': 'req-e3a7c77e-d7b1-4a88-af08-a1007094f5bc'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:31:06.723471 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-e3a7c77e-d7b1-4a88-af08-a1007094f5bc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6204c9e825f841aab905eac66e6a4a1e', 'request_id': 'req-e3a7c77e-d7b1-4a88-af08-a1007094f5bc'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:31:06.807728 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-e3a7c77e-d7b1-4a88-af08-a1007094f5bc admin admin] The token cache is being invalidate because project 6204c9e825f841aab905eac66e6a4a1e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:31:06.808142 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-e3a7c77e-d7b1-4a88-af08-a1007094f5bc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:31:06.808792 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-e3a7c77e-d7b1-4a88-af08-a1007094f5bc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:31:06.810333 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1080/2168] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 24 09:31:06 2022] DELETE /identity/v3/projects/6204c9e825f841aab905eac66e6a4a1e => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:31:06.816400 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd27f55b-d3e6-4c25-8400-0293d4294b7d None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:31:06.818990 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd27f55b-d3e6-4c25-8400-0293d4294b7d None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:31:06.818990 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd27f55b-d3e6-4c25-8400-0293d4294b7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:31:06.840426 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-cd27f55b-d3e6-4c25-8400-0293d4294b7d None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:31:06.841627 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-cd27f55b-d3e6-4c25-8400-0293d4294b7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:31:06.862912 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:31:06.905541 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1089/2169] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 09:31:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:31:06.914844 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd24ac0-9d70-4656-ae30-34716cf78277 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:31:06.915129 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd24ac0-9d70-4656-ae30-34716cf78277 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:31:06.915412 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd24ac0-9d70-4656-ae30-34716cf78277 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:31:06.938394 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-0cd24ac0-9d70-4656-ae30-34716cf78277 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:31:06.942951 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-0cd24ac0-9d70-4656-ae30-34716cf78277 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:31:06.955433 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1081/2170] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 09:31:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:31:06.956756 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd27f55b-d3e6-4c25-8400-0293d4294b7d None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:31:06.969323 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd27f55b-d3e6-4c25-8400-0293d4294b7d None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:31:06.975084 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45bf588-ee7e-464b-9f02-c55c4589a53d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:31:06.975735 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45bf588-ee7e-464b-9f02-c55c4589a53d admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:31:06.976014 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45bf588-ee7e-464b-9f02-c55c4589a53d admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:31:06.976285 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45bf588-ee7e-464b-9f02-c55c4589a53d admin admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:31:06.976934 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45bf588-ee7e-464b-9f02-c55c4589a53d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:31:06.977238 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45bf588-ee7e-464b-9f02-c55c4589a53d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:31:06.978831 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45bf588-ee7e-464b-9f02-c55c4589a53d admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:31:07.003890 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1090/2171] 10.222.0.16 () {58 vars in 1153 bytes} [Wed Aug 24 09:31:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:31:07.008539 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd24ac0-9d70-4656-ae30-34716cf78277 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:31:07.020337 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd24ac0-9d70-4656-ae30-34716cf78277 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:31:07.025811 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6dfd6d3-4153-48aa-a248-7fede4f2eb59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:31:07.026422 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dfd6d3-4153-48aa-a248-7fede4f2eb59 admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:31:07.026698 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dfd6d3-4153-48aa-a248-7fede4f2eb59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:31:07.026965 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dfd6d3-4153-48aa-a248-7fede4f2eb59 admin admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:31:07.027609 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6dfd6d3-4153-48aa-a248-7fede4f2eb59 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:31:07.030117 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6dfd6d3-4153-48aa-a248-7fede4f2eb59 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:31:07.042235 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:31:07.128642 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1082/2172] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 24 09:31:07 2022] POST /identity/v3/projects => generated 399 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:31:07.133004 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45bf588-ee7e-464b-9f02-c55c4589a53d admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:31:07.144967 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45bf588-ee7e-464b-9f02-c55c4589a53d admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:31:07.204714 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e5104b-8ed7-4b34-bab8-77dea49be4ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:31:07.205375 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e5104b-8ed7-4b34-bab8-77dea49be4ee admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:31:07.205752 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e5104b-8ed7-4b34-bab8-77dea49be4ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:31:07.206004 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e5104b-8ed7-4b34-bab8-77dea49be4ee admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:31:07.206948 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e5104b-8ed7-4b34-bab8-77dea49be4ee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:31:07.209327 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e5104b-8ed7-4b34-bab8-77dea49be4ee admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:31:07.319691 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1091/2173] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 24 09:31:07 2022] POST /identity/v3/users => generated 300 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:31:07.319822 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6dfd6d3-4153-48aa-a248-7fede4f2eb59 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:31:07.329009 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6dfd6d3-4153-48aa-a248-7fede4f2eb59 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:31:07.333729 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4fbb0a-a2ef-4fe1-8655-543d5a6d1587 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:31:07.334336 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4fbb0a-a2ef-4fe1-8655-543d5a6d1587 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:31:07.334621 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4fbb0a-a2ef-4fe1-8655-543d5a6d1587 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:31:07.334893 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4fbb0a-a2ef-4fe1-8655-543d5a6d1587 admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:31:07.335567 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4fbb0a-a2ef-4fe1-8655-543d5a6d1587 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:31:07.337459 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4fbb0a-a2ef-4fe1-8655-543d5a6d1587 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:31:07.348026 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1083/2174] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 24 09:31:07 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:31:07.352026 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e5104b-8ed7-4b34-bab8-77dea49be4ee admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:31:07.362663 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e5104b-8ed7-4b34-bab8-77dea49be4ee admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:31:07.367327 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2794bbc2-821d-4b59-bc1f-e6b37050695f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:31:07.367958 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2794bbc2-821d-4b59-bc1f-e6b37050695f admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:31:07.368240 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2794bbc2-821d-4b59-bc1f-e6b37050695f admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:31:07.368518 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2794bbc2-821d-4b59-bc1f-e6b37050695f admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:31:07.369190 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2794bbc2-821d-4b59-bc1f-e6b37050695f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:31:07.370772 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2794bbc2-821d-4b59-bc1f-e6b37050695f admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:31:07.381416 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1092/2175] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 24 09:31:07 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:31:07.385298 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4fbb0a-a2ef-4fe1-8655-543d5a6d1587 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:31:07.395003 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4fbb0a-a2ef-4fe1-8655-543d5a6d1587 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:31:07.399807 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68442628-168f-4bce-936f-5366373d543c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:31:07.400454 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68442628-168f-4bce-936f-5366373d543c admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:31:07.400733 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68442628-168f-4bce-936f-5366373d543c admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:31:07.401026 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68442628-168f-4bce-936f-5366373d543c admin admin] PATH_INFO: `/v3/projects/8cb7cd613d6d4575a1063c921b36db45/users/211be1de0e404cc0b85217c7e818dd4f/roles/136cdf78e00a426997dd4e37faff489b` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:31:07.416406 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68442628-168f-4bce-936f-5366373d543c admin admin] RBAC: Authorizing `identity:create_grant(project_id=8cb7cd613d6d4575a1063c921b36db45, user_id=211be1de0e404cc0b85217c7e818dd4f, role_id=136cdf78e00a426997dd4e37faff489b)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:31:07.419330 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68442628-168f-4bce-936f-5366373d543c admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:31:07.487523 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1084/2176] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 24 09:31:07 2022] PUT /identity/v3/projects/8cb7cd613d6d4575a1063c921b36db45/users/211be1de0e404cc0b85217c7e818dd4f/roles/136cdf78e00a426997dd4e37faff489b => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:31:07.497441 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed00c01f-e385-44a8-9fb6-cb979584ff8a None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:31:07.497782 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed00c01f-e385-44a8-9fb6-cb979584ff8a None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:31:07.498121 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed00c01f-e385-44a8-9fb6-cb979584ff8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:31:07.528125 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-ed00c01f-e385-44a8-9fb6-cb979584ff8a None None] MFA Rules not processed for user `211be1de0e404cc0b85217c7e818dd4f`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:31:07.529356 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-ed00c01f-e385-44a8-9fb6-cb979584ff8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:31:07.595642 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1093/2177] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 09:31:07 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:31:07.606905 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68442628-168f-4bce-936f-5366373d543c admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:31:07.608681 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-68442628-168f-4bce-936f-5366373d543c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:31:07.665108 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68442628-168f-4bce-936f-5366373d543c admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:31:07.668693 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e6bab4-00d1-4c6c-b499-20ba1abb530f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c32b83577c46488906fe409a790b24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:31:07.669309 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e6bab4-00d1-4c6c-b499-20ba1abb530f service cinder] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:31:07.669591 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e6bab4-00d1-4c6c-b499-20ba1abb530f service cinder] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:31:07.669866 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e6bab4-00d1-4c6c-b499-20ba1abb530f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:31:07.678022 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e6bab4-00d1-4c6c-b499-20ba1abb530f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:31:07.679440 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e6bab4-00d1-4c6c-b499-20ba1abb530f service cinder] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:31:07.688095 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1085/2178] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:31:07 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:31:17.592254 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=88,keystone:INSERT=3 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:31:17.684145 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=44,keystone:INSERT=2 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:31:22.914381 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed00c01f-e385-44a8-9fb6-cb979584ff8a None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:31:22.984514 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed00c01f-e385-44a8-9fb6-cb979584ff8a None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:31:22.991125 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aacfc5a-812c-4b09-a731-d9dae5c6a982 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:31:22.991801 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aacfc5a-812c-4b09-a731-d9dae5c6a982 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:31:22.992082 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aacfc5a-812c-4b09-a731-d9dae5c6a982 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:31:22.992353 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aacfc5a-812c-4b09-a731-d9dae5c6a982 admin admin] PATH_INFO: `/v3/users/211be1de0e404cc0b85217c7e818dd4f` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:31:22.993512 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aacfc5a-812c-4b09-a731-d9dae5c6a982 admin admin] RBAC: Authorizing `identity:delete_user(user_id=211be1de0e404cc0b85217c7e818dd4f)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:31:22.995421 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aacfc5a-812c-4b09-a731-d9dae5c6a982 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:31:23.198095 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-8aacfc5a-812c-4b09-a731-d9dae5c6a982 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '211be1de0e404cc0b85217c7e818dd4f'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:31:23.198665 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-8aacfc5a-812c-4b09-a731-d9dae5c6a982 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '211be1de0e404cc0b85217c7e818dd4f'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:31:23.263822 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-8aacfc5a-812c-4b09-a731-d9dae5c6a982 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '211be1de0e404cc0b85217c7e818dd4f'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:31:23.264402 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-8aacfc5a-812c-4b09-a731-d9dae5c6a982 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '211be1de0e404cc0b85217c7e818dd4f'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:31:23.278745 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-8aacfc5a-812c-4b09-a731-d9dae5c6a982 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '211be1de0e404cc0b85217c7e818dd4f'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:31:23.305070 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1094/2179] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 24 09:31:22 2022] DELETE /identity/v3/users/211be1de0e404cc0b85217c7e818dd4f => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:31:23.308735 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b62fab-9ff2-40d7-9058-83155da54e19 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:31:23.309136 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b62fab-9ff2-40d7-9058-83155da54e19 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:31:23.309368 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b62fab-9ff2-40d7-9058-83155da54e19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:31:23.338206 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-b7b62fab-9ff2-40d7-9058-83155da54e19 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:31:23.339524 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-b7b62fab-9ff2-40d7-9058-83155da54e19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:31:23.405480 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1086/2180] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 09:31:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:31:23.417582 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aacfc5a-812c-4b09-a731-d9dae5c6a982 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:31:23.419064 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-8aacfc5a-812c-4b09-a731-d9dae5c6a982 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:31:23.480611 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aacfc5a-812c-4b09-a731-d9dae5c6a982 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:31:23.484318 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fafdd99-6a26-413b-bf71-7b84894424fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e5260e798404e8bb12b25ca8ea97052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:31:23.484928 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fafdd99-6a26-413b-bf71-7b84894424fb service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:31:23.485211 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fafdd99-6a26-413b-bf71-7b84894424fb service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:31:23.485489 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fafdd99-6a26-413b-bf71-7b84894424fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:31:23.496169 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fafdd99-6a26-413b-bf71-7b84894424fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:31:23.497610 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fafdd99-6a26-413b-bf71-7b84894424fb service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:31:23.508225 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1095/2181] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:31:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:31:23.952522 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b62fab-9ff2-40d7-9058-83155da54e19 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:31:23.952835 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-b7b62fab-9ff2-40d7-9058-83155da54e19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:31:23.965602 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b62fab-9ff2-40d7-9058-83155da54e19 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:31:23.970474 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a699899-34d6-4e7f-a51e-9187ac676feb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:31:23.971412 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a699899-34d6-4e7f-a51e-9187ac676feb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:31:23.971769 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a699899-34d6-4e7f-a51e-9187ac676feb admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:31:23.972181 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a699899-34d6-4e7f-a51e-9187ac676feb admin admin] PATH_INFO: `/v3/projects/8cb7cd613d6d4575a1063c921b36db45` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:31:23.973419 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a699899-34d6-4e7f-a51e-9187ac676feb admin admin] RBAC: Authorizing `identity:delete_project(project_id=8cb7cd613d6d4575a1063c921b36db45)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:31:23.975431 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a699899-34d6-4e7f-a51e-9187ac676feb admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:31:24.090986 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-7a699899-34d6-4e7f-a51e-9187ac676feb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8cb7cd613d6d4575a1063c921b36db45', 'request_id': 'req-7a699899-34d6-4e7f-a51e-9187ac676feb'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:31:24.099698 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-7a699899-34d6-4e7f-a51e-9187ac676feb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8cb7cd613d6d4575a1063c921b36db45', 'request_id': 'req-7a699899-34d6-4e7f-a51e-9187ac676feb'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:31:24.105610 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-7a699899-34d6-4e7f-a51e-9187ac676feb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8cb7cd613d6d4575a1063c921b36db45', 'request_id': 'req-7a699899-34d6-4e7f-a51e-9187ac676feb'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:31:24.206339 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-7a699899-34d6-4e7f-a51e-9187ac676feb admin admin] The token cache is being invalidate because project 8cb7cd613d6d4575a1063c921b36db45 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:31:24.206627 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-7a699899-34d6-4e7f-a51e-9187ac676feb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:31:24.207305 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-7a699899-34d6-4e7f-a51e-9187ac676feb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:31:24.208812 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1087/2182] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 24 09:31:23 2022] DELETE /identity/v3/projects/8cb7cd613d6d4575a1063c921b36db45 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:31:24.214111 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4bc5dd-cda5-465a-bacf-de498220fb99 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:31:24.214579 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4bc5dd-cda5-465a-bacf-de498220fb99 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:31:24.214680 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4bc5dd-cda5-465a-bacf-de498220fb99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:31:24.234504 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-bb4bc5dd-cda5-465a-bacf-de498220fb99 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:31:24.235737 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-bb4bc5dd-cda5-465a-bacf-de498220fb99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:31:24.252727 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:31:24.293921 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1096/2183] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 09:31:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:31:24.302732 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453b3106-0bc4-48da-8e9e-971c3519c9c4 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:31:24.303018 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453b3106-0bc4-48da-8e9e-971c3519c9c4 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:31:24.303301 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453b3106-0bc4-48da-8e9e-971c3519c9c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:31:24.322605 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-453b3106-0bc4-48da-8e9e-971c3519c9c4 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:31:24.323862 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-453b3106-0bc4-48da-8e9e-971c3519c9c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:31:24.331977 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1088/2184] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 09:31:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:31:24.336267 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4bc5dd-cda5-465a-bacf-de498220fb99 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:31:24.346201 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4bc5dd-cda5-465a-bacf-de498220fb99 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:31:24.351945 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54e1ea8-c0c1-4821-8cce-d14ec7ef7b3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:31:24.352542 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54e1ea8-c0c1-4821-8cce-d14ec7ef7b3e admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:31:24.352821 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54e1ea8-c0c1-4821-8cce-d14ec7ef7b3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:31:24.353094 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54e1ea8-c0c1-4821-8cce-d14ec7ef7b3e admin admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:31:24.353741 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54e1ea8-c0c1-4821-8cce-d14ec7ef7b3e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:31:24.354047 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54e1ea8-c0c1-4821-8cce-d14ec7ef7b3e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:31:24.355653 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54e1ea8-c0c1-4821-8cce-d14ec7ef7b3e admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:31:24.367461 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1097/2185] 10.222.0.16 () {58 vars in 1153 bytes} [Wed Aug 24 09:31:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:31:24.371951 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453b3106-0bc4-48da-8e9e-971c3519c9c4 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:31:24.381319 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453b3106-0bc4-48da-8e9e-971c3519c9c4 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:31:24.385962 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7885131d-883c-4fe3-a9fc-8474122a04e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:31:24.386598 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7885131d-883c-4fe3-a9fc-8474122a04e6 admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:31:24.386876 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7885131d-883c-4fe3-a9fc-8474122a04e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:31:24.387141 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7885131d-883c-4fe3-a9fc-8474122a04e6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:31:24.387804 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7885131d-883c-4fe3-a9fc-8474122a04e6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:31:24.389712 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7885131d-883c-4fe3-a9fc-8474122a04e6 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:31:24.474956 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1089/2186] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 24 09:31:24 2022] POST /identity/v3/projects => generated 373 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:31:24.476295 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54e1ea8-c0c1-4821-8cce-d14ec7ef7b3e admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:31:24.486895 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54e1ea8-c0c1-4821-8cce-d14ec7ef7b3e admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:31:24.543619 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca093db-983c-4491-a48e-ac2546637021 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:31:24.544237 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca093db-983c-4491-a48e-ac2546637021 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:31:24.544521 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca093db-983c-4491-a48e-ac2546637021 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:31:24.544796 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca093db-983c-4491-a48e-ac2546637021 admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:31:24.545422 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca093db-983c-4491-a48e-ac2546637021 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:31:24.547350 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca093db-983c-4491-a48e-ac2546637021 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:31:24.648421 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1098/2187] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 24 09:31:24 2022] POST /identity/v3/users => generated 287 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:31:24.652570 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7885131d-883c-4fe3-a9fc-8474122a04e6 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:31:24.664348 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7885131d-883c-4fe3-a9fc-8474122a04e6 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:31:24.669003 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebadea2-c23f-4905-b554-926ff3742985 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:31:24.669601 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebadea2-c23f-4905-b554-926ff3742985 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:31:24.669881 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebadea2-c23f-4905-b554-926ff3742985 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:31:24.670151 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebadea2-c23f-4905-b554-926ff3742985 admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:31:24.670814 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebadea2-c23f-4905-b554-926ff3742985 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:31:24.672431 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebadea2-c23f-4905-b554-926ff3742985 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:31:24.682288 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1090/2188] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 24 09:31:24 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:31:24.688281 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca093db-983c-4491-a48e-ac2546637021 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:31:24.701312 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca093db-983c-4491-a48e-ac2546637021 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:31:24.706112 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e5bc42-1b15-4944-ae69-e640375f1dc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:31:24.707242 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e5bc42-1b15-4944-ae69-e640375f1dc9 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:31:24.707528 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e5bc42-1b15-4944-ae69-e640375f1dc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:31:24.707830 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e5bc42-1b15-4944-ae69-e640375f1dc9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:31:24.708476 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e5bc42-1b15-4944-ae69-e640375f1dc9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:31:24.710078 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e5bc42-1b15-4944-ae69-e640375f1dc9 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:31:24.721236 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1099/2189] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 24 09:31:24 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:31:24.726051 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebadea2-c23f-4905-b554-926ff3742985 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:31:24.735877 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebadea2-c23f-4905-b554-926ff3742985 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:31:24.741475 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3de08b-e51a-43ba-bef6-47fab93313e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:31:24.742323 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3de08b-e51a-43ba-bef6-47fab93313e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:31:24.742604 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3de08b-e51a-43ba-bef6-47fab93313e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:31:24.743001 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3de08b-e51a-43ba-bef6-47fab93313e7 admin admin] PATH_INFO: `/v3/projects/4934cbcf5e1448fc826068c0779252b3/users/e09379db7745459e94c671589ba2f6fd/roles/136cdf78e00a426997dd4e37faff489b` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:31:24.758664 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3de08b-e51a-43ba-bef6-47fab93313e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4934cbcf5e1448fc826068c0779252b3, user_id=e09379db7745459e94c671589ba2f6fd, role_id=136cdf78e00a426997dd4e37faff489b)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:31:24.761547 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3de08b-e51a-43ba-bef6-47fab93313e7 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:31:24.853063 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1091/2190] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 24 09:31:24 2022] PUT /identity/v3/projects/4934cbcf5e1448fc826068c0779252b3/users/e09379db7745459e94c671589ba2f6fd/roles/136cdf78e00a426997dd4e37faff489b => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:31:24.858146 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7006c7a3-e934-4e88-94e3-adb4566c855c None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:31:24.858479 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7006c7a3-e934-4e88-94e3-adb4566c855c None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:31:24.858753 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7006c7a3-e934-4e88-94e3-adb4566c855c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:31:24.877285 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-7006c7a3-e934-4e88-94e3-adb4566c855c None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:31:24.878466 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-7006c7a3-e934-4e88-94e3-adb4566c855c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:31:24.943489 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1100/2191] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 09:31:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:31:24.953157 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3de08b-e51a-43ba-bef6-47fab93313e7 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:31:24.954879 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-cd3de08b-e51a-43ba-bef6-47fab93313e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:31:24.977737 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:31:25.014413 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3de08b-e51a-43ba-bef6-47fab93313e7 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:31:25.018147 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a686891f-f8d6-4f82-b274-ede32727701f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e5260e798404e8bb12b25ca8ea97052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:31:25.018719 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a686891f-f8d6-4f82-b274-ede32727701f service neutron] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:31:25.018976 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a686891f-f8d6-4f82-b274-ede32727701f service neutron] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:31:25.019297 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a686891f-f8d6-4f82-b274-ede32727701f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:31:25.029448 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a686891f-f8d6-4f82-b274-ede32727701f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:31:25.030901 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a686891f-f8d6-4f82-b274-ede32727701f service neutron] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:31:25.041124 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1092/2192] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:31:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:31:30.521797 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df631dc-81da-48ff-839c-477f1b059c04 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:31:30.522262 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df631dc-81da-48ff-839c-477f1b059c04 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:31:30.522262 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df631dc-81da-48ff-839c-477f1b059c04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:31:30.559075 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-9df631dc-81da-48ff-839c-477f1b059c04 None None] MFA Rules not processed for user `e09379db7745459e94c671589ba2f6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:31:30.560738 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-9df631dc-81da-48ff-839c-477f1b059c04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:31:30.600067 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:31:30.648689 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1101/2193] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 09:31:30 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:31:30.653492 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a686891f-f8d6-4f82-b274-ede32727701f service neutron] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:31:30.663606 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a686891f-f8d6-4f82-b274-ede32727701f service neutron] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:31:30.668641 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f85977c-a76b-498b-b42d-466d1ba2027d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:31:30.673617 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f85977c-a76b-498b-b42d-466d1ba2027d admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:31:30.673617 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f85977c-a76b-498b-b42d-466d1ba2027d admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:31:30.673617 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f85977c-a76b-498b-b42d-466d1ba2027d admin admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:31:30.673617 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f85977c-a76b-498b-b42d-466d1ba2027d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:31:30.673617 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f85977c-a76b-498b-b42d-466d1ba2027d admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:31:30.762478 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1093/2194] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 24 09:31:30 2022] POST /identity/v3/projects => generated 373 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:31:30.767054 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df631dc-81da-48ff-839c-477f1b059c04 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:31:30.779171 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df631dc-81da-48ff-839c-477f1b059c04 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:31:30.841855 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdccadd9-7f14-4daf-9acf-82b2e610ef8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:31:30.842492 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdccadd9-7f14-4daf-9acf-82b2e610ef8a admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:31:30.842771 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdccadd9-7f14-4daf-9acf-82b2e610ef8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:31:30.843039 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdccadd9-7f14-4daf-9acf-82b2e610ef8a admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:31:30.843710 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdccadd9-7f14-4daf-9acf-82b2e610ef8a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:31:30.845614 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdccadd9-7f14-4daf-9acf-82b2e610ef8a admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:31:30.944295 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1102/2195] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 24 09:31:30 2022] POST /identity/v3/users => generated 293 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:31:30.948667 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f85977c-a76b-498b-b42d-466d1ba2027d admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:31:30.959376 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f85977c-a76b-498b-b42d-466d1ba2027d admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:31:30.964594 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68a27df-23d2-4bd6-acb9-489ab946606e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:31:30.965216 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68a27df-23d2-4bd6-acb9-489ab946606e admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:31:30.965594 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68a27df-23d2-4bd6-acb9-489ab946606e admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:31:30.965818 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68a27df-23d2-4bd6-acb9-489ab946606e admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:31:30.966447 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68a27df-23d2-4bd6-acb9-489ab946606e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:31:30.968129 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68a27df-23d2-4bd6-acb9-489ab946606e admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:31:30.980434 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1094/2196] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 24 09:31:30 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:31:30.984777 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdccadd9-7f14-4daf-9acf-82b2e610ef8a admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:31:30.998023 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdccadd9-7f14-4daf-9acf-82b2e610ef8a admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:31:31.004172 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e9b86f-fef5-4a79-9eb6-83119d5052b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:31:31.005736 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e9b86f-fef5-4a79-9eb6-83119d5052b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:31:31.006019 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e9b86f-fef5-4a79-9eb6-83119d5052b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:31:31.006295 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e9b86f-fef5-4a79-9eb6-83119d5052b4 admin admin] PATH_INFO: `/v3/projects/9e5646913af8418989bb8adafc34c301/users/9188f0454e9146dfa169c31dc7d8642f/roles/15f634d4a2d247ddab90c458e5e1c141` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:31:31.022759 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e9b86f-fef5-4a79-9eb6-83119d5052b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e5646913af8418989bb8adafc34c301, user_id=9188f0454e9146dfa169c31dc7d8642f, role_id=15f634d4a2d247ddab90c458e5e1c141)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:31:31.025662 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e9b86f-fef5-4a79-9eb6-83119d5052b4 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:31:31.076016 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1103/2197] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 24 09:31:30 2022] PUT /identity/v3/projects/9e5646913af8418989bb8adafc34c301/users/9188f0454e9146dfa169c31dc7d8642f/roles/15f634d4a2d247ddab90c458e5e1c141 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:31:31.080197 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68a27df-23d2-4bd6-acb9-489ab946606e admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:31:31.091310 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68a27df-23d2-4bd6-acb9-489ab946606e admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:31:31.147761 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e271504-c6b9-4420-801b-cbf82ba9ca90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:31:31.148392 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e271504-c6b9-4420-801b-cbf82ba9ca90 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:31:31.148671 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e271504-c6b9-4420-801b-cbf82ba9ca90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:31:31.148945 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e271504-c6b9-4420-801b-cbf82ba9ca90 admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:31:31.149578 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e271504-c6b9-4420-801b-cbf82ba9ca90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:31:31.151165 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e271504-c6b9-4420-801b-cbf82ba9ca90 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:31:31.161325 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1095/2198] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 24 09:31:31 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:31:31.165898 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e9b86f-fef5-4a79-9eb6-83119d5052b4 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:31:31.176836 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e9b86f-fef5-4a79-9eb6-83119d5052b4 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:31:31.181605 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d0ade4-899e-4c0c-95ca-4379b14d0746 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:31:31.182264 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d0ade4-899e-4c0c-95ca-4379b14d0746 admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:31:31.182542 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d0ade4-899e-4c0c-95ca-4379b14d0746 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:31:31.182817 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d0ade4-899e-4c0c-95ca-4379b14d0746 admin admin] PATH_INFO: `/v3/projects/9e5646913af8418989bb8adafc34c301/users/9188f0454e9146dfa169c31dc7d8642f/roles/15f634d4a2d247ddab90c458e5e1c141` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:31:31.184740 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d0ade4-899e-4c0c-95ca-4379b14d0746 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e5646913af8418989bb8adafc34c301, user_id=9188f0454e9146dfa169c31dc7d8642f, role_id=15f634d4a2d247ddab90c458e5e1c141)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:31:31.187369 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d0ade4-899e-4c0c-95ca-4379b14d0746 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:31:31.198338 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1104/2199] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 24 09:31:31 2022] PUT /identity/v3/projects/9e5646913af8418989bb8adafc34c301/users/9188f0454e9146dfa169c31dc7d8642f/roles/15f634d4a2d247ddab90c458e5e1c141 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:31:37.017372 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89999b4-9383-4e22-80ea-126073de7f4e None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:31:37.017869 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89999b4-9383-4e22-80ea-126073de7f4e None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:31:37.017984 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89999b4-9383-4e22-80ea-126073de7f4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:31:37.053347 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-a89999b4-9383-4e22-80ea-126073de7f4e None None] MFA Rules not processed for user `9188f0454e9146dfa169c31dc7d8642f`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:31:37.054995 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-a89999b4-9383-4e22-80ea-126073de7f4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:31:37.141094 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1096/2200] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 09:31:37 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:31:37.156132 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d0ade4-899e-4c0c-95ca-4379b14d0746 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:31:37.157954 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-99d0ade4-899e-4c0c-95ca-4379b14d0746 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:31:37.246420 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d0ade4-899e-4c0c-95ca-4379b14d0746 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:31:37.252146 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a2eeff-1a49-442a-8392-27d679206781 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f055f7dc51d43bda9b315ca67f11d58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:31:37.252772 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a2eeff-1a49-442a-8392-27d679206781 service nova] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:31:37.253052 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a2eeff-1a49-442a-8392-27d679206781 service nova] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:31:37.253358 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a2eeff-1a49-442a-8392-27d679206781 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:31:37.259126 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:31:37.333134 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a2eeff-1a49-442a-8392-27d679206781 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:31:37.335033 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a2eeff-1a49-442a-8392-27d679206781 service nova] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:31:37.348695 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1105/2201] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:31:37 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:31:39.912821 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021fb369-13e0-485e-8959-ac5757609fc9 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:31:39.913303 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021fb369-13e0-485e-8959-ac5757609fc9 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:31:39.913382 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021fb369-13e0-485e-8959-ac5757609fc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:31:39.933718 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-021fb369-13e0-485e-8959-ac5757609fc9 None None] MFA Rules not processed for user `e09379db7745459e94c671589ba2f6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:31:39.934966 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-021fb369-13e0-485e-8959-ac5757609fc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:31:39.942308 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1097/2202] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 09:31:39 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:31:47.341902 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=24 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:31:49.928060 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=97,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:31:52.101472 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2a43a9-3f24-4982-9bed-3a776d77cf8b None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:31:52.102031 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2a43a9-3f24-4982-9bed-3a776d77cf8b None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:31:52.104692 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2a43a9-3f24-4982-9bed-3a776d77cf8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:31:52.124399 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-9a2a43a9-3f24-4982-9bed-3a776d77cf8b None None] MFA Rules not processed for user `0aff36cf71984238b8a9b998043ec7be`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:31:52.125845 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-9a2a43a9-3f24-4982-9bed-3a776d77cf8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:31:52.185821 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1106/2203] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 09:31:52 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:31:55.134010 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021fb369-13e0-485e-8959-ac5757609fc9 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:31:55.139691 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-021fb369-13e0-485e-8959-ac5757609fc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:31:55.236957 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021fb369-13e0-485e-8959-ac5757609fc9 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:31:55.242493 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05decc2-efb4-4b6f-af76-49b1234481fa service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80cada22039d4f11935412ee578acd05', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:31:55.243135 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05decc2-efb4-4b6f-af76-49b1234481fa service placement] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:31:55.243423 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05decc2-efb4-4b6f-af76-49b1234481fa service placement] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:31:55.243726 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05decc2-efb4-4b6f-af76-49b1234481fa service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:31:55.245496 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-f05decc2-efb4-4b6f-af76-49b1234481fa service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:31:55.257662 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05decc2-efb4-4b6f-af76-49b1234481fa service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:31:55.259391 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05decc2-efb4-4b6f-af76-49b1234481fa service placement] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:31:55.270924 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1098/2204] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:31:55 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:32:02.178922 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=25 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:32:05.265560 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=32 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:32:50.116374 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2a43a9-3f24-4982-9bed-3a776d77cf8b None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:32:50.173097 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2a43a9-3f24-4982-9bed-3a776d77cf8b None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:32:50.176771 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9178a189-ca61-4307-a0a4-ca21aeda981c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e5260e798404e8bb12b25ca8ea97052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:32:50.177411 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9178a189-ca61-4307-a0a4-ca21aeda981c service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:32:50.177881 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9178a189-ca61-4307-a0a4-ca21aeda981c service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:32:50.178166 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9178a189-ca61-4307-a0a4-ca21aeda981c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:32:50.180109 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-9178a189-ca61-4307-a0a4-ca21aeda981c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:32:50.194089 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9178a189-ca61-4307-a0a4-ca21aeda981c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:32:50.195497 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9178a189-ca61-4307-a0a4-ca21aeda981c service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:32:50.207254 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1107/2205] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:32:50 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:32:51.383774 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05decc2-efb4-4b6f-af76-49b1234481fa service placement] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:32:51.385231 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-f05decc2-efb4-4b6f-af76-49b1234481fa service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:32:51.397940 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05decc2-efb4-4b6f-af76-49b1234481fa service placement] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:32:51.400784 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd3f778-8fad-431a-84fe-1d792d650b09 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e5260e798404e8bb12b25ca8ea97052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:32:51.406233 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd3f778-8fad-431a-84fe-1d792d650b09 service neutron] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:32:51.406233 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd3f778-8fad-431a-84fe-1d792d650b09 service neutron] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:32:51.406233 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd3f778-8fad-431a-84fe-1d792d650b09 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:32:51.406233 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-9bd3f778-8fad-431a-84fe-1d792d650b09 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:32:51.418242 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd3f778-8fad-431a-84fe-1d792d650b09 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:32:51.419753 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd3f778-8fad-431a-84fe-1d792d650b09 service neutron] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:32:51.430951 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1099/2206] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:32:51 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:32:59.667491 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9178a189-ca61-4307-a0a4-ca21aeda981c service neutron] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:32:59.680745 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9178a189-ca61-4307-a0a4-ca21aeda981c service neutron] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:32:59.694424 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6224075a-0321-45f8-a534-5952c2065675 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e5260e798404e8bb12b25ca8ea97052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:32:59.695159 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6224075a-0321-45f8-a534-5952c2065675 service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:32:59.695442 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6224075a-0321-45f8-a534-5952c2065675 service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:32:59.695748 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6224075a-0321-45f8-a534-5952c2065675 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:32:59.698207 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-6224075a-0321-45f8-a534-5952c2065675 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:32:59.776792 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6224075a-0321-45f8-a534-5952c2065675 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:32:59.778239 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6224075a-0321-45f8-a534-5952c2065675 service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:32:59.791130 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1108/2207] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:32:59 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:33:01.426719 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=6 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:33:02.246217 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478eba2d-097c-4a03-b93a-0934377a2a93 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:33:02.246511 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478eba2d-097c-4a03-b93a-0934377a2a93 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:33:02.246791 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478eba2d-097c-4a03-b93a-0934377a2a93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:33:02.266368 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-478eba2d-097c-4a03-b93a-0934377a2a93 None None] MFA Rules not processed for user `def04335e57e48409a6943fa231d47c3`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:33:02.267726 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-478eba2d-097c-4a03-b93a-0934377a2a93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:33:02.275327 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1100/2208] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 09:33:02 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:33:05.459536 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cb1451-baa1-435d-9551-8d69ac26c771 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:33:05.460055 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cb1451-baa1-435d-9551-8d69ac26c771 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:33:05.460147 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cb1451-baa1-435d-9551-8d69ac26c771 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:33:05.482082 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-08cb1451-baa1-435d-9551-8d69ac26c771 None None] MFA Rules not processed for user `0489462287fc4aea8e8e720af24e1ef8`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:33:05.484253 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-08cb1451-baa1-435d-9551-8d69ac26c771 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:33:05.540459 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1109/2209] 10.222.0.16 () {58 vars in 1009 bytes} [Wed Aug 24 09:33:05 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:33:05.746034 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478eba2d-097c-4a03-b93a-0934377a2a93 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:33:05.747532 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-478eba2d-097c-4a03-b93a-0934377a2a93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:33:05.825914 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478eba2d-097c-4a03-b93a-0934377a2a93 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:33:05.833048 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da922fab-eb75-4c16-901c-a2406fc0c5cd service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdd5f0d67fe54e9fae6f4af644988d93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:33:05.833849 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da922fab-eb75-4c16-901c-a2406fc0c5cd service swift] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:33:05.834308 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da922fab-eb75-4c16-901c-a2406fc0c5cd service swift] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:33:05.834752 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da922fab-eb75-4c16-901c-a2406fc0c5cd service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:33:05.845073 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da922fab-eb75-4c16-901c-a2406fc0c5cd service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:33:05.846697 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da922fab-eb75-4c16-901c-a2406fc0c5cd service swift] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:33:05.855473 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1101/2210] 10.222.0.16 () {60 vars in 1384 bytes} [Wed Aug 24 09:33:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 09:33:06.950552 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cb1451-baa1-435d-9551-8d69ac26c771 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:33:06.979790 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cb1451-baa1-435d-9551-8d69ac26c771 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:33:06.979790 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522d44fa-03ac-4c89-9ed8-35384071e52a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f055f7dc51d43bda9b315ca67f11d58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:33:06.979790 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522d44fa-03ac-4c89-9ed8-35384071e52a service nova] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:33:06.979790 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522d44fa-03ac-4c89-9ed8-35384071e52a service nova] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:33:06.980226 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522d44fa-03ac-4c89-9ed8-35384071e52a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:33:06.980902 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-522d44fa-03ac-4c89-9ed8-35384071e52a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:33:07.057325 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:33:07.078900 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522d44fa-03ac-4c89-9ed8-35384071e52a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:33:07.080633 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522d44fa-03ac-4c89-9ed8-35384071e52a service nova] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:33:07.092068 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1110/2211] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:33:06 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:33:08.999165 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da922fab-eb75-4c16-901c-a2406fc0c5cd service swift] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:33:09.001152 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-da922fab-eb75-4c16-901c-a2406fc0c5cd service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:33:09.076032 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da922fab-eb75-4c16-901c-a2406fc0c5cd service swift] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:33:09.091707 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d45b5e-0126-43d9-96bf-1bc2d3c40c61 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c32b83577c46488906fe409a790b24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:33:09.091707 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d45b5e-0126-43d9-96bf-1bc2d3c40c61 service cinder] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:33:09.091707 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d45b5e-0126-43d9-96bf-1bc2d3c40c61 service cinder] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:33:09.091707 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d45b5e-0126-43d9-96bf-1bc2d3c40c61 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:33:09.092097 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-f8d45b5e-0126-43d9-96bf-1bc2d3c40c61 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:33:09.170472 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d45b5e-0126-43d9-96bf-1bc2d3c40c61 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:33:09.172281 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d45b5e-0126-43d9-96bf-1bc2d3c40c61 service cinder] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:33:09.184014 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1102/2212] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:33:08 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:33:17.141753 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=7 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:33:19.178402 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=83 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:33:20.636259 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522d44fa-03ac-4c89-9ed8-35384071e52a service nova] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:33:20.645381 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522d44fa-03ac-4c89-9ed8-35384071e52a service nova] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:33:20.648976 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e728c81-a53e-4c8c-9644-05bed9c9979e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e5260e798404e8bb12b25ca8ea97052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:33:20.649572 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e728c81-a53e-4c8c-9644-05bed9c9979e service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:33:20.649852 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e728c81-a53e-4c8c-9644-05bed9c9979e service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:33:20.650127 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e728c81-a53e-4c8c-9644-05bed9c9979e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:33:20.651897 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-6e728c81-a53e-4c8c-9644-05bed9c9979e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:33:20.661033 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e728c81-a53e-4c8c-9644-05bed9c9979e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:33:20.662440 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e728c81-a53e-4c8c-9644-05bed9c9979e service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:33:20.671400 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1111/2213] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:33:20 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:33:26.057090 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d45b5e-0126-43d9-96bf-1bc2d3c40c61 service cinder] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:33:26.066214 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d45b5e-0126-43d9-96bf-1bc2d3c40c61 service cinder] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:33:26.069888 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81437d06-efea-46d4-ae34-19f717060033 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e5260e798404e8bb12b25ca8ea97052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:33:26.074428 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81437d06-efea-46d4-ae34-19f717060033 service neutron] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:33:26.074428 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81437d06-efea-46d4-ae34-19f717060033 service neutron] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:33:26.074428 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81437d06-efea-46d4-ae34-19f717060033 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:33:26.074428 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-81437d06-efea-46d4-ae34-19f717060033 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:33:26.082604 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81437d06-efea-46d4-ae34-19f717060033 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:33:26.084095 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81437d06-efea-46d4-ae34-19f717060033 service neutron] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:33:26.104145 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1103/2214] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:33:26 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:33:26.504426 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5968ae-41a3-4b89-b4ff-571efb58a98f None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:33:26.504964 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5968ae-41a3-4b89-b4ff-571efb58a98f None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:33:26.504964 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5968ae-41a3-4b89-b4ff-571efb58a98f None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:33:26.506407 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1112/2215] 10.222.0.16 () {54 vars in 921 bytes} [Wed Aug 24 09:33:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:33:26.512044 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a02e74-7e27-49ae-bf51-9c78831964ad None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:33:26.512335 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a02e74-7e27-49ae-bf51-9c78831964ad None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:33:26.512625 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a02e74-7e27-49ae-bf51-9c78831964ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:33:26.536197 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-b7a02e74-7e27-49ae-bf51-9c78831964ad None None] MFA Rules not processed for user `1f055f7dc51d43bda9b315ca67f11d58`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:33:26.537545 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-b7a02e74-7e27-49ae-bf51-9c78831964ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:33:26.548643 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1104/2216] 10.222.0.16 () {58 vars in 1018 bytes} [Wed Aug 24 09:33:26 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:33:26.560588 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5968ae-41a3-4b89-b4ff-571efb58a98f None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:33:26.562364 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-3e5968ae-41a3-4b89-b4ff-571efb58a98f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:33:26.619760 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5968ae-41a3-4b89-b4ff-571efb58a98f None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:33:26.623258 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0a4e63-76cd-489b-a91b-b7bdf308a464 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea2666d430a646d386104addf1f42888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:33:26.623880 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0a4e63-76cd-489b-a91b-b7bdf308a464 service glance] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:33:26.624160 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0a4e63-76cd-489b-a91b-b7bdf308a464 service glance] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:33:26.624429 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0a4e63-76cd-489b-a91b-b7bdf308a464 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:33:26.634522 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0a4e63-76cd-489b-a91b-b7bdf308a464 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:33:26.635974 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0a4e63-76cd-489b-a91b-b7bdf308a464 service glance] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:33:26.646396 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1113/2217] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:33:26 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:33:26.840527 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ad3544-da35-4a98-9c8d-3f54266d3f0c None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:33:26.840814 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ad3544-da35-4a98-9c8d-3f54266d3f0c None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:33:26.841093 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ad3544-da35-4a98-9c8d-3f54266d3f0c None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:33:26.842524 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1105/2218] 10.222.0.16 () {54 vars in 921 bytes} [Wed Aug 24 09:33:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:33:26.847152 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b701d70-a1ce-40fe-adf8-849fa0f809e6 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:33:26.847505 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b701d70-a1ce-40fe-adf8-849fa0f809e6 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:33:26.847807 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b701d70-a1ce-40fe-adf8-849fa0f809e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:33:26.866808 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-9b701d70-a1ce-40fe-adf8-849fa0f809e6 None None] MFA Rules not processed for user `8e5260e798404e8bb12b25ca8ea97052`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:33:26.868055 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-9b701d70-a1ce-40fe-adf8-849fa0f809e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:33:26.874940 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1114/2219] 10.222.0.16 () {58 vars in 1018 bytes} [Wed Aug 24 09:33:26 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:33:26.887176 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ad3544-da35-4a98-9c8d-3f54266d3f0c None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:33:26.898017 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ad3544-da35-4a98-9c8d-3f54266d3f0c None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:33:26.901494 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7211e7-7368-4ecf-8145-e8ccd410ca9c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e5260e798404e8bb12b25ca8ea97052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:33:26.902082 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7211e7-7368-4ecf-8145-e8ccd410ca9c service neutron] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:33:26.902357 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7211e7-7368-4ecf-8145-e8ccd410ca9c service neutron] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:33:26.902624 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7211e7-7368-4ecf-8145-e8ccd410ca9c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:33:26.909841 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7211e7-7368-4ecf-8145-e8ccd410ca9c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:33:26.911326 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7211e7-7368-4ecf-8145-e8ccd410ca9c service neutron] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:33:26.919069 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1106/2220] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:33:26 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 32 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:33:27.001933 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d309ae66-daa9-47a7-b6ef-2785a82b7a2b None None] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:33:27.002214 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d309ae66-daa9-47a7-b6ef-2785a82b7a2b None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:33:27.002484 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d309ae66-daa9-47a7-b6ef-2785a82b7a2b None None] PATH_INFO: `/` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:33:27.003898 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1115/2221] 10.222.0.16 () {54 vars in 921 bytes} [Wed Aug 24 09:33:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 09:33:27.009434 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33810c86-1a6e-4b2c-a249-d079413f5288 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:33:27.010200 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33810c86-1a6e-4b2c-a249-d079413f5288 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:33:27.010480 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33810c86-1a6e-4b2c-a249-d079413f5288 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:33:27.032816 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-33810c86-1a6e-4b2c-a249-d079413f5288 None None] MFA Rules not processed for user `1f055f7dc51d43bda9b315ca67f11d58`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:33:27.034051 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-33810c86-1a6e-4b2c-a249-d079413f5288 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:33:27.043745 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1107/2222] 10.222.0.16 () {58 vars in 1018 bytes} [Wed Aug 24 09:33:27 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:33:27.054333 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d309ae66-daa9-47a7-b6ef-2785a82b7a2b None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:33:27.065788 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d309ae66-daa9-47a7-b6ef-2785a82b7a2b None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:33:27.069429 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9078f21c-306e-41a1-b55e-92b9f12868ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e5260e798404e8bb12b25ca8ea97052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:33:27.070198 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9078f21c-306e-41a1-b55e-92b9f12868ba service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:33:27.070474 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9078f21c-306e-41a1-b55e-92b9f12868ba service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:33:27.070745 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9078f21c-306e-41a1-b55e-92b9f12868ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:33:27.082010 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9078f21c-306e-41a1-b55e-92b9f12868ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:33:27.083421 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9078f21c-306e-41a1-b55e-92b9f12868ba service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:33:27.094437 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1116/2223] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:33:27 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:33:27.156067 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33810c86-1a6e-4b2c-a249-d079413f5288 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:33:27.167704 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33810c86-1a6e-4b2c-a249-d079413f5288 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:33:27.172631 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb52ea2a-4901-47c3-84bb-515775f7f2f9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80cada22039d4f11935412ee578acd05', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:33:27.173285 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb52ea2a-4901-47c3-84bb-515775f7f2f9 service placement] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:33:27.173564 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb52ea2a-4901-47c3-84bb-515775f7f2f9 service placement] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:33:27.173831 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb52ea2a-4901-47c3-84bb-515775f7f2f9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:33:27.175501 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-cb52ea2a-4901-47c3-84bb-515775f7f2f9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:33:27.187034 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb52ea2a-4901-47c3-84bb-515775f7f2f9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:33:27.188625 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb52ea2a-4901-47c3-84bb-515775f7f2f9 service placement] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:33:27.198171 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1108/2224] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:33:27 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:33:37.089760 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=41 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:33:37.194026 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=24 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:34:55.897229 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9078f21c-306e-41a1-b55e-92b9f12868ba service neutron] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:34:55.898801 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-9078f21c-306e-41a1-b55e-92b9f12868ba service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:34:55.976191 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9078f21c-306e-41a1-b55e-92b9f12868ba service neutron] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:34:55.982788 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6847e48-c228-4fe6-88b1-e27d71cbdcf4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f055f7dc51d43bda9b315ca67f11d58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:34:55.984107 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6847e48-c228-4fe6-88b1-e27d71cbdcf4 service nova] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:34:55.984736 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6847e48-c228-4fe6-88b1-e27d71cbdcf4 service nova] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:34:55.985077 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6847e48-c228-4fe6-88b1-e27d71cbdcf4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:34:55.987019 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-f6847e48-c228-4fe6-88b1-e27d71cbdcf4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:34:56.000070 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6847e48-c228-4fe6-88b1-e27d71cbdcf4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:34:56.001885 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6847e48-c228-4fe6-88b1-e27d71cbdcf4 service nova] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:34:56.013519 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1117/2225] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:34:55 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:34:56.253396 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb52ea2a-4901-47c3-84bb-515775f7f2f9 service placement] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:34:56.305032 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb52ea2a-4901-47c3-84bb-515775f7f2f9 service placement] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:34:56.310480 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3ad24b-c952-4877-a691-5138da5d40fc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c32b83577c46488906fe409a790b24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:34:56.311365 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3ad24b-c952-4877-a691-5138da5d40fc service cinder] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:34:56.311716 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3ad24b-c952-4877-a691-5138da5d40fc service cinder] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:34:56.312954 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3ad24b-c952-4877-a691-5138da5d40fc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:34:56.314870 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-ce3ad24b-c952-4877-a691-5138da5d40fc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:34:56.329634 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3ad24b-c952-4877-a691-5138da5d40fc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:34:56.329634 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3ad24b-c952-4877-a691-5138da5d40fc service cinder] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:34:56.340172 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1109/2226] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:34:56 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:35:06.008407 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=28 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:35:06.335218 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=18 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:35:32.436717 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6847e48-c228-4fe6-88b1-e27d71cbdcf4 service nova] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:35:32.438153 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-f6847e48-c228-4fe6-88b1-e27d71cbdcf4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:35:32.462563 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6847e48-c228-4fe6-88b1-e27d71cbdcf4 service nova] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:35:32.466057 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde719a5-e75d-4033-b4de-856ae68c1ab1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c32b83577c46488906fe409a790b24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:35:32.466649 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde719a5-e75d-4033-b4de-856ae68c1ab1 service cinder] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:35:32.466929 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde719a5-e75d-4033-b4de-856ae68c1ab1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:35:32.467203 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde719a5-e75d-4033-b4de-856ae68c1ab1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:35:32.469613 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-cde719a5-e75d-4033-b4de-856ae68c1ab1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:35:32.527226 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde719a5-e75d-4033-b4de-856ae68c1ab1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:35:32.528859 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde719a5-e75d-4033-b4de-856ae68c1ab1 service cinder] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:35:32.538851 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1118/2227] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:35:32 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:35:42.534639 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=33 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:35:52.376930 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3ad24b-c952-4877-a691-5138da5d40fc service cinder] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:35:52.385417 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3ad24b-c952-4877-a691-5138da5d40fc service cinder] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:35:52.389107 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3325ad46-1c4e-4e10-a2f0-5eb8e61eae88 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c32b83577c46488906fe409a790b24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:35:52.389707 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3325ad46-1c4e-4e10-a2f0-5eb8e61eae88 service cinder] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:35:52.390031 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3325ad46-1c4e-4e10-a2f0-5eb8e61eae88 service cinder] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:35:52.390333 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3325ad46-1c4e-4e10-a2f0-5eb8e61eae88 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:35:52.392097 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-3325ad46-1c4e-4e10-a2f0-5eb8e61eae88 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:35:52.450014 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3325ad46-1c4e-4e10-a2f0-5eb8e61eae88 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:35:52.451357 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3325ad46-1c4e-4e10-a2f0-5eb8e61eae88 service cinder] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:35:52.460222 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1110/2228] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:35:52 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:36:02.456164 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=28 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:36:37.842437 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde719a5-e75d-4033-b4de-856ae68c1ab1 service cinder] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:36:37.851604 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde719a5-e75d-4033-b4de-856ae68c1ab1 service cinder] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:36:37.855240 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bfb7e2-f0ea-48a7-ac85-f08b54a29647 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c32b83577c46488906fe409a790b24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:36:37.856242 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bfb7e2-f0ea-48a7-ac85-f08b54a29647 service cinder] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:36:37.856530 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bfb7e2-f0ea-48a7-ac85-f08b54a29647 service cinder] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:36:37.856800 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bfb7e2-f0ea-48a7-ac85-f08b54a29647 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:36:37.865089 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bfb7e2-f0ea-48a7-ac85-f08b54a29647 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:36:37.866501 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bfb7e2-f0ea-48a7-ac85-f08b54a29647 service cinder] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:36:37.876196 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1119/2229] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:36:37 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:36:47.871297 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=6 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:37:05.176736 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3325ad46-1c4e-4e10-a2f0-5eb8e61eae88 service cinder] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:37:05.178112 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-3325ad46-1c4e-4e10-a2f0-5eb8e61eae88 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:37:05.213604 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3325ad46-1c4e-4e10-a2f0-5eb8e61eae88 service cinder] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:37:05.218573 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24508e3-59d3-4e5a-a3f7-6a633d847f9d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80cada22039d4f11935412ee578acd05', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:37:05.219164 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24508e3-59d3-4e5a-a3f7-6a633d847f9d service placement] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:37:05.219444 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24508e3-59d3-4e5a-a3f7-6a633d847f9d service placement] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:37:05.219749 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24508e3-59d3-4e5a-a3f7-6a633d847f9d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:37:05.228609 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24508e3-59d3-4e5a-a3f7-6a633d847f9d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:37:05.230157 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24508e3-59d3-4e5a-a3f7-6a633d847f9d service placement] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:37:05.240462 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1111/2230] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:37:05 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:37:15.236176 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=15 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:37:41.733632 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bfb7e2-f0ea-48a7-ac85-f08b54a29647 service cinder] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:37:41.752849 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bfb7e2-f0ea-48a7-ac85-f08b54a29647 service cinder] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:37:41.756524 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13136446-50fe-48f6-970f-77fbbbe06e73 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e5260e798404e8bb12b25ca8ea97052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:37:41.757122 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13136446-50fe-48f6-970f-77fbbbe06e73 service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:37:41.757402 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13136446-50fe-48f6-970f-77fbbbe06e73 service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:37:41.757672 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13136446-50fe-48f6-970f-77fbbbe06e73 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:37:41.830112 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13136446-50fe-48f6-970f-77fbbbe06e73 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:37:41.831458 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13136446-50fe-48f6-970f-77fbbbe06e73 service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:37:41.850301 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1120/2231] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:37:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:37:45.541979 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24508e3-59d3-4e5a-a3f7-6a633d847f9d service placement] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:37:45.553015 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24508e3-59d3-4e5a-a3f7-6a633d847f9d service placement] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:37:45.558763 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644f85b7-b6cd-4a42-8e61-080aa7689668 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80cada22039d4f11935412ee578acd05', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:37:45.559381 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644f85b7-b6cd-4a42-8e61-080aa7689668 service placement] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:37:45.559723 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644f85b7-b6cd-4a42-8e61-080aa7689668 service placement] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:37:45.560003 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644f85b7-b6cd-4a42-8e61-080aa7689668 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:37:45.562318 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-644f85b7-b6cd-4a42-8e61-080aa7689668 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:37:45.575449 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644f85b7-b6cd-4a42-8e61-080aa7689668 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:37:45.577815 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644f85b7-b6cd-4a42-8e61-080aa7689668 service placement] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:37:45.590945 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1112/2232] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:37:45 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:37:49.515904 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13136446-50fe-48f6-970f-77fbbbe06e73 service neutron] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:37:49.527161 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13136446-50fe-48f6-970f-77fbbbe06e73 service neutron] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:37:49.532821 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0f9ada-c00e-417a-8071-98c379b4226c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80cada22039d4f11935412ee578acd05', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:37:49.533421 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0f9ada-c00e-417a-8071-98c379b4226c service placement] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:37:49.533701 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0f9ada-c00e-417a-8071-98c379b4226c service placement] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:37:49.533975 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0f9ada-c00e-417a-8071-98c379b4226c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:37:49.535876 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-fc0f9ada-c00e-417a-8071-98c379b4226c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:37:49.548835 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0f9ada-c00e-417a-8071-98c379b4226c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:37:49.550727 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0f9ada-c00e-417a-8071-98c379b4226c service placement] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:37:49.562849 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1121/2233] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:37:49 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:37:49.678141 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644f85b7-b6cd-4a42-8e61-080aa7689668 service placement] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:37:49.694916 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644f85b7-b6cd-4a42-8e61-080aa7689668 service placement] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:37:49.699387 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92c9afd-e3ac-47ff-9952-667baf1a1ac0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:37:49.700045 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92c9afd-e3ac-47ff-9952-667baf1a1ac0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:37:49.700324 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92c9afd-e3ac-47ff-9952-667baf1a1ac0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:37:49.700692 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92c9afd-e3ac-47ff-9952-667baf1a1ac0 admin admin] PATH_INFO: `/v3/users/e09379db7745459e94c671589ba2f6fd` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:37:49.701837 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92c9afd-e3ac-47ff-9952-667baf1a1ac0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e09379db7745459e94c671589ba2f6fd)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:37:49.704058 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92c9afd-e3ac-47ff-9952-667baf1a1ac0 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:37:49.852778 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-a92c9afd-e3ac-47ff-9952-667baf1a1ac0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e09379db7745459e94c671589ba2f6fd'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:37:49.853474 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-a92c9afd-e3ac-47ff-9952-667baf1a1ac0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e09379db7745459e94c671589ba2f6fd'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:37:49.865640 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-a92c9afd-e3ac-47ff-9952-667baf1a1ac0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e09379db7745459e94c671589ba2f6fd'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:37:49.872674 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-a92c9afd-e3ac-47ff-9952-667baf1a1ac0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e09379db7745459e94c671589ba2f6fd'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:37:49.928826 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-a92c9afd-e3ac-47ff-9952-667baf1a1ac0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e09379db7745459e94c671589ba2f6fd'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:37:49.936351 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1113/2234] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 24 09:37:49 2022] DELETE /identity/v3/users/e09379db7745459e94c671589ba2f6fd => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:37:50.200684 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0f9ada-c00e-417a-8071-98c379b4226c service placement] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:37:50.212889 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0f9ada-c00e-417a-8071-98c379b4226c service placement] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:37:50.280958 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d410c1e0-0aa0-475a-bb16-77dc9d0524f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:37:50.281599 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d410c1e0-0aa0-475a-bb16-77dc9d0524f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:37:50.281934 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d410c1e0-0aa0-475a-bb16-77dc9d0524f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:37:50.282161 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d410c1e0-0aa0-475a-bb16-77dc9d0524f4 admin admin] PATH_INFO: `/v3/projects/4934cbcf5e1448fc826068c0779252b3` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:37:50.283297 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d410c1e0-0aa0-475a-bb16-77dc9d0524f4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4934cbcf5e1448fc826068c0779252b3)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:37:50.285255 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d410c1e0-0aa0-475a-bb16-77dc9d0524f4 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:37:50.385586 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-d410c1e0-0aa0-475a-bb16-77dc9d0524f4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4934cbcf5e1448fc826068c0779252b3', 'request_id': 'req-d410c1e0-0aa0-475a-bb16-77dc9d0524f4'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:37:50.392982 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-d410c1e0-0aa0-475a-bb16-77dc9d0524f4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4934cbcf5e1448fc826068c0779252b3', 'request_id': 'req-d410c1e0-0aa0-475a-bb16-77dc9d0524f4'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:37:50.395125 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:37:50.399039 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-d410c1e0-0aa0-475a-bb16-77dc9d0524f4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4934cbcf5e1448fc826068c0779252b3', 'request_id': 'req-d410c1e0-0aa0-475a-bb16-77dc9d0524f4'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:37:50.511283 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-d410c1e0-0aa0-475a-bb16-77dc9d0524f4 admin admin] The token cache is being invalidate because project 4934cbcf5e1448fc826068c0779252b3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81602) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:37:50.511573 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-d410c1e0-0aa0-475a-bb16-77dc9d0524f4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:37:50.512219 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-d410c1e0-0aa0-475a-bb16-77dc9d0524f4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:37:50.513695 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1122/2235] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 24 09:37:50 2022] DELETE /identity/v3/projects/4934cbcf5e1448fc826068c0779252b3 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:37:50.517759 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92c9afd-e3ac-47ff-9952-667baf1a1ac0 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:37:50.519242 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-a92c9afd-e3ac-47ff-9952-667baf1a1ac0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:37:50.533171 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92c9afd-e3ac-47ff-9952-667baf1a1ac0 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:37:50.590440 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d352df0-0df1-4d77-988f-86ba4337761f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:37:50.591065 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d352df0-0df1-4d77-988f-86ba4337761f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:37:50.591344 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d352df0-0df1-4d77-988f-86ba4337761f admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:37:50.592158 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d352df0-0df1-4d77-988f-86ba4337761f admin admin] PATH_INFO: `/v3/users/9188f0454e9146dfa169c31dc7d8642f` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:37:50.593261 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d352df0-0df1-4d77-988f-86ba4337761f admin admin] RBAC: Authorizing `identity:delete_user(user_id=9188f0454e9146dfa169c31dc7d8642f)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:37:50.595473 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d352df0-0df1-4d77-988f-86ba4337761f admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:37:50.781096 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-1d352df0-0df1-4d77-988f-86ba4337761f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9188f0454e9146dfa169c31dc7d8642f'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:37:50.781750 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-1d352df0-0df1-4d77-988f-86ba4337761f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9188f0454e9146dfa169c31dc7d8642f'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:37:50.792099 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=81,keystone:DELETE=18,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:37:50.793964 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-1d352df0-0df1-4d77-988f-86ba4337761f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9188f0454e9146dfa169c31dc7d8642f'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:37:50.800939 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-1d352df0-0df1-4d77-988f-86ba4337761f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9188f0454e9146dfa169c31dc7d8642f'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:37:50.802954 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d410c1e0-0aa0-475a-bb16-77dc9d0524f4 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:37:50.805297 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-d410c1e0-0aa0-475a-bb16-77dc9d0524f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:37:50.881963 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d410c1e0-0aa0-475a-bb16-77dc9d0524f4 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:37:50.885816 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c83523-c424-4098-93d0-8407c9006cba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e5260e798404e8bb12b25ca8ea97052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:37:50.886428 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c83523-c424-4098-93d0-8407c9006cba service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:37:50.886710 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c83523-c424-4098-93d0-8407c9006cba service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:37:50.887013 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c83523-c424-4098-93d0-8407c9006cba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:37:50.889304 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-f2c83523-c424-4098-93d0-8407c9006cba service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:37:50.899944 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-1d352df0-0df1-4d77-988f-86ba4337761f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9188f0454e9146dfa169c31dc7d8642f'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:37:50.907008 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1114/2236] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 24 09:37:50 2022] DELETE /identity/v3/users/9188f0454e9146dfa169c31dc7d8642f => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:37:50.984589 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c83523-c424-4098-93d0-8407c9006cba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:37:50.986066 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c83523-c424-4098-93d0-8407c9006cba service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:37:51.007477 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1123/2237] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:37:50 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 09:37:51.195070 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d352df0-0df1-4d77-988f-86ba4337761f admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:37:51.196571 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-1d352df0-0df1-4d77-988f-86ba4337761f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:37:51.265060 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d352df0-0df1-4d77-988f-86ba4337761f admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:37:51.270165 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab72768e-cfdf-4d1f-adae-a3993155d7c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:37:51.270794 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab72768e-cfdf-4d1f-adae-a3993155d7c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:37:51.271075 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab72768e-cfdf-4d1f-adae-a3993155d7c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:37:51.271348 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab72768e-cfdf-4d1f-adae-a3993155d7c2 admin admin] PATH_INFO: `/v3/projects/9e5646913af8418989bb8adafc34c301` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:37:51.272502 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab72768e-cfdf-4d1f-adae-a3993155d7c2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9e5646913af8418989bb8adafc34c301)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:37:51.274413 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab72768e-cfdf-4d1f-adae-a3993155d7c2 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:37:51.368145 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-ab72768e-cfdf-4d1f-adae-a3993155d7c2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9e5646913af8418989bb8adafc34c301', 'request_id': 'req-ab72768e-cfdf-4d1f-adae-a3993155d7c2'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:37:51.374839 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-ab72768e-cfdf-4d1f-adae-a3993155d7c2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9e5646913af8418989bb8adafc34c301', 'request_id': 'req-ab72768e-cfdf-4d1f-adae-a3993155d7c2'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:37:51.380601 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-ab72768e-cfdf-4d1f-adae-a3993155d7c2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9e5646913af8418989bb8adafc34c301', 'request_id': 'req-ab72768e-cfdf-4d1f-adae-a3993155d7c2'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:37:51.455439 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-ab72768e-cfdf-4d1f-adae-a3993155d7c2 admin admin] The token cache is being invalidate because project 9e5646913af8418989bb8adafc34c301 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:37:51.455810 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-ab72768e-cfdf-4d1f-adae-a3993155d7c2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:37:51.456517 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-ab72768e-cfdf-4d1f-adae-a3993155d7c2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:37:51.458064 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1115/2238] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 24 09:37:51 2022] DELETE /identity/v3/projects/9e5646913af8418989bb8adafc34c301 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:37:56.180619 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c83523-c424-4098-93d0-8407c9006cba service neutron] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:37:56.182015 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-f2c83523-c424-4098-93d0-8407c9006cba service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:37:56.222966 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:37:56.237883 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c83523-c424-4098-93d0-8407c9006cba service neutron] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:37:56.285626 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721143ae-9f64-41f1-964e-4870f06ecced service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c32b83577c46488906fe409a790b24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:37:56.286229 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721143ae-9f64-41f1-964e-4870f06ecced service cinder] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:37:56.286518 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721143ae-9f64-41f1-964e-4870f06ecced service cinder] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:37:56.286795 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721143ae-9f64-41f1-964e-4870f06ecced service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:37:56.288614 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-721143ae-9f64-41f1-964e-4870f06ecced service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:37:56.360096 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721143ae-9f64-41f1-964e-4870f06ecced service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:37:56.361455 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721143ae-9f64-41f1-964e-4870f06ecced service cinder] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:37:56.372278 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1124/2239] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:37:56 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:38:00.769111 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab72768e-cfdf-4d1f-adae-a3993155d7c2 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:38:00.770596 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-ab72768e-cfdf-4d1f-adae-a3993155d7c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:38:00.787362 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab72768e-cfdf-4d1f-adae-a3993155d7c2 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:38:00.792961 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14cd8a9-36dc-4e1b-b78d-ddf365b3587f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f055f7dc51d43bda9b315ca67f11d58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:38:00.793586 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14cd8a9-36dc-4e1b-b78d-ddf365b3587f service nova] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:38:00.793908 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14cd8a9-36dc-4e1b-b78d-ddf365b3587f service nova] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:38:00.794220 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14cd8a9-36dc-4e1b-b78d-ddf365b3587f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:38:00.796705 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-f14cd8a9-36dc-4e1b-b78d-ddf365b3587f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:38:00.864129 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:DELETE=9,keystone:SELECT=89,keystone:INSERT=2 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:38:00.876618 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14cd8a9-36dc-4e1b-b78d-ddf365b3587f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:38:00.878231 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14cd8a9-36dc-4e1b-b78d-ddf365b3587f service nova] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:38:00.887198 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1116/2240] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:38:00 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:38:06.367469 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=57 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:38:07.538113 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721143ae-9f64-41f1-964e-4870f06ecced service cinder] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:38:07.551431 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721143ae-9f64-41f1-964e-4870f06ecced service cinder] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:38:07.559110 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903e97ea-1caf-4be2-9922-b074571efc40 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f055f7dc51d43bda9b315ca67f11d58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:38:07.559915 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903e97ea-1caf-4be2-9922-b074571efc40 service nova] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:38:07.560276 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903e97ea-1caf-4be2-9922-b074571efc40 service nova] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:38:07.561133 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903e97ea-1caf-4be2-9922-b074571efc40 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:38:07.562973 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-903e97ea-1caf-4be2-9922-b074571efc40 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:38:07.644915 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903e97ea-1caf-4be2-9922-b074571efc40 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:38:07.646507 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903e97ea-1caf-4be2-9922-b074571efc40 service nova] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:38:07.664682 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1125/2241] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:38:07 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:38:09.128552 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14cd8a9-36dc-4e1b-b78d-ddf365b3587f service nova] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:38:09.130083 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-f14cd8a9-36dc-4e1b-b78d-ddf365b3587f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:38:09.184399 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14cd8a9-36dc-4e1b-b78d-ddf365b3587f service nova] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:38:09.188111 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c834392-e3a3-479f-bef6-fc3817a00700 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e5260e798404e8bb12b25ca8ea97052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:38:09.188716 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c834392-e3a3-479f-bef6-fc3817a00700 service neutron] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:38:09.188990 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c834392-e3a3-479f-bef6-fc3817a00700 service neutron] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:38:09.189258 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c834392-e3a3-479f-bef6-fc3817a00700 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:38:09.190933 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-1c834392-e3a3-479f-bef6-fc3817a00700 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:38:09.268038 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c834392-e3a3-479f-bef6-fc3817a00700 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:38:09.269479 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c834392-e3a3-479f-bef6-fc3817a00700 service neutron] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:38:09.278127 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1117/2242] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:38:09 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:38:10.760569 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f88bed5-68e4-430d-b39f-96f793e25518 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:38:10.765504 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f88bed5-68e4-430d-b39f-96f793e25518 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:38:10.765504 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f88bed5-68e4-430d-b39f-96f793e25518 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:38:10.788711 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-0f88bed5-68e4-430d-b39f-96f793e25518 None None] MFA Rules not processed for user `b9099dc3ad92431cbac98f144690038c`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:38:10.789912 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-0f88bed5-68e4-430d-b39f-96f793e25518 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:38:10.805806 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1126/2243] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 09:38:10 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:38:11.115866 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c834392-e3a3-479f-bef6-fc3817a00700 service neutron] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:38:11.117253 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-1c834392-e3a3-479f-bef6-fc3817a00700 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:38:11.128393 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c834392-e3a3-479f-bef6-fc3817a00700 service neutron] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:38:11.132018 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3549ff60-6f10-4d14-bc7f-d228ba19ce0b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e5260e798404e8bb12b25ca8ea97052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:38:11.137537 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3549ff60-6f10-4d14-bc7f-d228ba19ce0b service neutron] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:38:11.137537 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3549ff60-6f10-4d14-bc7f-d228ba19ce0b service neutron] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:38:11.137537 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3549ff60-6f10-4d14-bc7f-d228ba19ce0b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:38:11.137537 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-3549ff60-6f10-4d14-bc7f-d228ba19ce0b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:38:11.143791 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3549ff60-6f10-4d14-bc7f-d228ba19ce0b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:38:11.145201 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3549ff60-6f10-4d14-bc7f-d228ba19ce0b service neutron] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:38:11.152996 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1118/2244] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:38:11 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:38:20.802948 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=51 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:38:21.149471 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=69 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:38:21.945868 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f88bed5-68e4-430d-b39f-96f793e25518 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:38:21.945868 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-0f88bed5-68e4-430d-b39f-96f793e25518 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:38:21.960135 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f88bed5-68e4-430d-b39f-96f793e25518 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:38:21.966708 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6a0364-60da-484a-9613-08ee2c817adc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f055f7dc51d43bda9b315ca67f11d58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:38:21.967319 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6a0364-60da-484a-9613-08ee2c817adc service nova] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:38:21.967622 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6a0364-60da-484a-9613-08ee2c817adc service nova] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:38:21.967905 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6a0364-60da-484a-9613-08ee2c817adc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:38:21.969664 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-6c6a0364-60da-484a-9613-08ee2c817adc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:38:21.982034 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6a0364-60da-484a-9613-08ee2c817adc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:38:21.983682 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6a0364-60da-484a-9613-08ee2c817adc service nova] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:38:21.994287 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1127/2245] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:38:21 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:38:22.421161 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3549ff60-6f10-4d14-bc7f-d228ba19ce0b service neutron] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:38:22.429881 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3549ff60-6f10-4d14-bc7f-d228ba19ce0b service neutron] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:38:22.433619 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d860dc1-457f-4b5b-94a8-de300ab18f67 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e5260e798404e8bb12b25ca8ea97052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:38:22.434221 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d860dc1-457f-4b5b-94a8-de300ab18f67 service neutron] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:38:22.436750 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d860dc1-457f-4b5b-94a8-de300ab18f67 service neutron] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:38:22.436750 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d860dc1-457f-4b5b-94a8-de300ab18f67 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:38:22.437112 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-3d860dc1-457f-4b5b-94a8-de300ab18f67 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:38:22.446507 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d860dc1-457f-4b5b-94a8-de300ab18f67 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:38:22.447952 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d860dc1-457f-4b5b-94a8-de300ab18f67 service neutron] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:38:22.455829 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1119/2246] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:38:22 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:38:26.875424 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6a0364-60da-484a-9613-08ee2c817adc service nova] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:38:26.886709 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6a0364-60da-484a-9613-08ee2c817adc service nova] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:38:26.893025 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c631fbf1-72b2-476d-9246-154e76456e29 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f055f7dc51d43bda9b315ca67f11d58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:38:26.893638 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c631fbf1-72b2-476d-9246-154e76456e29 service nova] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:38:26.893918 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c631fbf1-72b2-476d-9246-154e76456e29 service nova] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:38:26.894189 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c631fbf1-72b2-476d-9246-154e76456e29 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:38:26.896054 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-c631fbf1-72b2-476d-9246-154e76456e29 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:38:26.909106 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c631fbf1-72b2-476d-9246-154e76456e29 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:38:26.910684 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c631fbf1-72b2-476d-9246-154e76456e29 service nova] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:38:26.922565 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1128/2247] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:38:26 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:38:32.452261 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=6 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:38:36.917545 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=12 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:39:27.824462 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7082d5bf-acf5-431e-b4f4-09c12a61da92 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:39:27.824972 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7082d5bf-acf5-431e-b4f4-09c12a61da92 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:39:27.825175 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7082d5bf-acf5-431e-b4f4-09c12a61da92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:39:27.888215 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-7082d5bf-acf5-431e-b4f4-09c12a61da92 None None] MFA Rules not processed for user `0489462287fc4aea8e8e720af24e1ef8`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:39:27.889394 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-7082d5bf-acf5-431e-b4f4-09c12a61da92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:39:27.956083 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1120/2248] 10.222.0.16 () {58 vars in 967 bytes} [Wed Aug 24 09:39:27 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:39:28.100877 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c631fbf1-72b2-476d-9246-154e76456e29 service nova] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:39:28.102320 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-c631fbf1-72b2-476d-9246-154e76456e29 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:39:28.191092 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c631fbf1-72b2-476d-9246-154e76456e29 service nova] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:39:28.195955 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb110cb9-f538-4947-9faf-b92488cf57d9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdd5f0d67fe54e9fae6f4af644988d93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:39:28.196556 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb110cb9-f538-4947-9faf-b92488cf57d9 service swift] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:39:28.196840 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb110cb9-f538-4947-9faf-b92488cf57d9 service swift] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:39:28.197114 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb110cb9-f538-4947-9faf-b92488cf57d9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:39:28.205072 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb110cb9-f538-4947-9faf-b92488cf57d9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:39:28.206642 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb110cb9-f538-4947-9faf-b92488cf57d9 service swift] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:39:28.214808 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1129/2249] 10.222.0.16 () {60 vars in 1384 bytes} [Wed Aug 24 09:39:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 09:39:30.232703 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3add93-909f-4d33-be48-1643a082af5d None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:39:30.233252 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3add93-909f-4d33-be48-1643a082af5d None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:39:30.233252 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3add93-909f-4d33-be48-1643a082af5d None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:39:30.234583 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1121/2250] 10.222.0.16 () {54 vars in 912 bytes} [Wed Aug 24 09:39:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:39:30.240423 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b83bc79-ee29-4ca4-9a15-de3a19b6eaa3 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:39:30.240912 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b83bc79-ee29-4ca4-9a15-de3a19b6eaa3 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:39:30.241103 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b83bc79-ee29-4ca4-9a15-de3a19b6eaa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:39:30.289783 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-5b83bc79-ee29-4ca4-9a15-de3a19b6eaa3 None None] MFA Rules not processed for user `80cada22039d4f11935412ee578acd05`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:39:30.290991 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-5b83bc79-ee29-4ca4-9a15-de3a19b6eaa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:39:30.374035 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1130/2251] 10.222.0.16 () {58 vars in 1009 bytes} [Wed Aug 24 09:39:30 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:39:30.398859 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3add93-909f-4d33-be48-1643a082af5d None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:39:30.400713 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-0b3add93-909f-4d33-be48-1643a082af5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:39:30.419685 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3add93-909f-4d33-be48-1643a082af5d None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:39:30.424860 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44818589-f0d3-47cc-afc1-3b80596e061e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80cada22039d4f11935412ee578acd05', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:39:30.425527 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44818589-f0d3-47cc-afc1-3b80596e061e service placement] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:39:30.425809 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44818589-f0d3-47cc-afc1-3b80596e061e service placement] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:39:30.426081 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44818589-f0d3-47cc-afc1-3b80596e061e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:39:30.435345 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44818589-f0d3-47cc-afc1-3b80596e061e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:39:30.436966 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44818589-f0d3-47cc-afc1-3b80596e061e service placement] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:39:30.450275 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1122/2252] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:39:30 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:39:34.518656 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e8106e-7fe7-47e9-892a-564f48576910 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:39:34.519202 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e8106e-7fe7-47e9-892a-564f48576910 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:39:34.519202 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e8106e-7fe7-47e9-892a-564f48576910 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:39:34.546753 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-f0e8106e-7fe7-47e9-892a-564f48576910 None None] MFA Rules not processed for user `cbcaa84bc9c54d7aa4178d0174a5936e`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:39:34.548565 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-f0e8106e-7fe7-47e9-892a-564f48576910 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:39:34.555544 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1131/2253] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 09:39:34 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:39:36.258453 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44818589-f0d3-47cc-afc1-3b80596e061e service placement] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:39:36.259916 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-44818589-f0d3-47cc-afc1-3b80596e061e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:39:36.273592 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44818589-f0d3-47cc-afc1-3b80596e061e service placement] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:39:36.278545 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d59e635-6eae-41dd-98c1-6c22aeddbcb5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80cada22039d4f11935412ee578acd05', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:39:36.279139 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d59e635-6eae-41dd-98c1-6c22aeddbcb5 service placement] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:39:36.279423 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d59e635-6eae-41dd-98c1-6c22aeddbcb5 service placement] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:39:36.279744 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d59e635-6eae-41dd-98c1-6c22aeddbcb5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:39:36.281403 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-2d59e635-6eae-41dd-98c1-6c22aeddbcb5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:39:36.292368 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d59e635-6eae-41dd-98c1-6c22aeddbcb5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:39:36.293928 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d59e635-6eae-41dd-98c1-6c22aeddbcb5 service placement] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:39:36.306062 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1123/2254] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:39:36 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:39:44.544359 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=86 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:39:46.300135 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=56 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:39:56.592345 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e8106e-7fe7-47e9-892a-564f48576910 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:39:56.594359 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-f0e8106e-7fe7-47e9-892a-564f48576910 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:39:56.605980 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e8106e-7fe7-47e9-892a-564f48576910 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:39:56.610184 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eaf3dd0-764e-4351-986c-6c0a073b9031 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c32b83577c46488906fe409a790b24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:39:56.610791 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaf3dd0-764e-4351-986c-6c0a073b9031 service cinder] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:39:56.611073 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaf3dd0-764e-4351-986c-6c0a073b9031 service cinder] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:39:56.611347 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaf3dd0-764e-4351-986c-6c0a073b9031 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:39:56.613143 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-8eaf3dd0-764e-4351-986c-6c0a073b9031 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:39:56.693446 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eaf3dd0-764e-4351-986c-6c0a073b9031 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:39:56.694872 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eaf3dd0-764e-4351-986c-6c0a073b9031 service cinder] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:39:56.713770 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1132/2255] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:39:56 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:40:06.711136 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=41 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:40:32.470254 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d59e635-6eae-41dd-98c1-6c22aeddbcb5 service placement] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:40:32.481007 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d59e635-6eae-41dd-98c1-6c22aeddbcb5 service placement] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:40:32.484700 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8f49e0-1ff0-416e-819e-84fa4d0776f9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c32b83577c46488906fe409a790b24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:40:32.485555 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8f49e0-1ff0-416e-819e-84fa4d0776f9 service cinder] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:40:32.485841 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8f49e0-1ff0-416e-819e-84fa4d0776f9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:40:32.486117 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8f49e0-1ff0-416e-819e-84fa4d0776f9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:40:32.487906 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-ea8f49e0-1ff0-416e-819e-84fa4d0776f9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:40:32.570211 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8f49e0-1ff0-416e-819e-84fa4d0776f9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:40:32.571696 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8f49e0-1ff0-416e-819e-84fa4d0776f9 service cinder] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:40:32.581226 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1124/2256] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:40:32 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:40:42.576493 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=37 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:40:48.761625 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eaf3dd0-764e-4351-986c-6c0a073b9031 service cinder] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:40:48.774382 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eaf3dd0-764e-4351-986c-6c0a073b9031 service cinder] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:40:48.779924 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f36274-ccb7-4383-8f6f-b4ee961bb552 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f055f7dc51d43bda9b315ca67f11d58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:40:48.780547 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f36274-ccb7-4383-8f6f-b4ee961bb552 service nova] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:40:48.780817 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f36274-ccb7-4383-8f6f-b4ee961bb552 service nova] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:40:48.781091 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f36274-ccb7-4383-8f6f-b4ee961bb552 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:40:48.783451 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-c1f36274-ccb7-4383-8f6f-b4ee961bb552 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:40:48.796927 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f36274-ccb7-4383-8f6f-b4ee961bb552 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:40:48.799480 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f36274-ccb7-4383-8f6f-b4ee961bb552 service nova] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:40:48.812405 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1133/2257] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:40:48 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:40:51.679328 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8f49e0-1ff0-416e-819e-84fa4d0776f9 service cinder] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:40:51.687885 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8f49e0-1ff0-416e-819e-84fa4d0776f9 service cinder] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:40:51.692294 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4902e5ac-cf30-425c-86e4-72a0fc876505 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c32b83577c46488906fe409a790b24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:40:51.692924 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4902e5ac-cf30-425c-86e4-72a0fc876505 service cinder] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:40:51.693205 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4902e5ac-cf30-425c-86e4-72a0fc876505 service cinder] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:40:51.693474 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4902e5ac-cf30-425c-86e4-72a0fc876505 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:40:51.695254 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-4902e5ac-cf30-425c-86e4-72a0fc876505 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:40:51.764864 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4902e5ac-cf30-425c-86e4-72a0fc876505 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:40:51.766289 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4902e5ac-cf30-425c-86e4-72a0fc876505 service cinder] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:40:51.776044 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1125/2258] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:40:51 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:40:58.807595 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=6 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:41:01.771819 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=32 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:42:06.180163 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f36274-ccb7-4383-8f6f-b4ee961bb552 service nova] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:42:06.192385 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f36274-ccb7-4383-8f6f-b4ee961bb552 service nova] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:42:06.197478 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce8837b-3541-45d1-9def-e4bbed83dc5f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80cada22039d4f11935412ee578acd05', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:42:06.198081 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce8837b-3541-45d1-9def-e4bbed83dc5f service placement] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:42:06.198364 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce8837b-3541-45d1-9def-e4bbed83dc5f service placement] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:42:06.198638 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce8837b-3541-45d1-9def-e4bbed83dc5f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:42:06.200476 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-3ce8837b-3541-45d1-9def-e4bbed83dc5f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:42:06.212615 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce8837b-3541-45d1-9def-e4bbed83dc5f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:42:06.214181 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce8837b-3541-45d1-9def-e4bbed83dc5f service placement] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:42:06.223887 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1134/2259] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:42:06 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 09:42:16.219673 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=6 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:43:01.916795 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4902e5ac-cf30-425c-86e4-72a0fc876505 service cinder] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:43:01.918234 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-4902e5ac-cf30-425c-86e4-72a0fc876505 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:43:01.929401 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4902e5ac-cf30-425c-86e4-72a0fc876505 service cinder] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:43:01.932998 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd081a7f-99d2-48dd-8b53-868e30de8f7f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c32b83577c46488906fe409a790b24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:43:01.933587 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd081a7f-99d2-48dd-8b53-868e30de8f7f service cinder] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:43:01.933866 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd081a7f-99d2-48dd-8b53-868e30de8f7f service cinder] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:43:01.934135 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd081a7f-99d2-48dd-8b53-868e30de8f7f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:43:01.942201 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd081a7f-99d2-48dd-8b53-868e30de8f7f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:43:01.943615 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd081a7f-99d2-48dd-8b53-868e30de8f7f service cinder] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:43:01.961961 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1126/2260] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:43:01 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:43:07.282149 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce8837b-3541-45d1-9def-e4bbed83dc5f service placement] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:43:07.295004 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce8837b-3541-45d1-9def-e4bbed83dc5f service placement] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:43:07.300317 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4e17cf-2232-4eda-9723-d4cf11bc618b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c32b83577c46488906fe409a790b24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:43:07.301198 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4e17cf-2232-4eda-9723-d4cf11bc618b service cinder] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:43:07.301676 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4e17cf-2232-4eda-9723-d4cf11bc618b service cinder] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:43:07.302162 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4e17cf-2232-4eda-9723-d4cf11bc618b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:43:07.312172 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4e17cf-2232-4eda-9723-d4cf11bc618b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:43:07.314120 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4e17cf-2232-4eda-9723-d4cf11bc618b service cinder] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:43:07.323991 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1135/2261] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:43:07 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:43:09.284615 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd081a7f-99d2-48dd-8b53-868e30de8f7f service cinder] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:43:09.293247 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd081a7f-99d2-48dd-8b53-868e30de8f7f service cinder] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:43:09.300959 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af07573-4a77-4184-8bec-f99a0c391020 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c32b83577c46488906fe409a790b24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:43:09.300959 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af07573-4a77-4184-8bec-f99a0c391020 service cinder] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:43:09.300959 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af07573-4a77-4184-8bec-f99a0c391020 service cinder] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:43:09.300959 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af07573-4a77-4184-8bec-f99a0c391020 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:43:09.307001 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af07573-4a77-4184-8bec-f99a0c391020 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:43:09.308444 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af07573-4a77-4184-8bec-f99a0c391020 service cinder] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:43:09.317789 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1127/2262] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:43:09 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:43:17.319429 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=6 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:43:19.313188 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=16 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:43:23.042292 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4e17cf-2232-4eda-9723-d4cf11bc618b service cinder] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:43:23.043802 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-be4e17cf-2232-4eda-9723-d4cf11bc618b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:43:23.122443 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4e17cf-2232-4eda-9723-d4cf11bc618b service cinder] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:43:23.127502 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9349736b-74b7-458c-9b79-a76898eb56dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:43:23.128163 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9349736b-74b7-458c-9b79-a76898eb56dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:43:23.128445 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9349736b-74b7-458c-9b79-a76898eb56dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:43:23.128724 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9349736b-74b7-458c-9b79-a76898eb56dc admin admin] PATH_INFO: `/v3/users/da3da89b69264fe98f264c861c215186` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:43:23.142284 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9349736b-74b7-458c-9b79-a76898eb56dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=da3da89b69264fe98f264c861c215186)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:43:23.144238 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9349736b-74b7-458c-9b79-a76898eb56dc admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:43:23.299593 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-9349736b-74b7-458c-9b79-a76898eb56dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'da3da89b69264fe98f264c861c215186'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:43:23.300300 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-9349736b-74b7-458c-9b79-a76898eb56dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'da3da89b69264fe98f264c861c215186'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:43:23.362977 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-9349736b-74b7-458c-9b79-a76898eb56dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'da3da89b69264fe98f264c861c215186'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:43:23.363631 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-9349736b-74b7-458c-9b79-a76898eb56dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'da3da89b69264fe98f264c861c215186'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:43:23.370844 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-9349736b-74b7-458c-9b79-a76898eb56dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'da3da89b69264fe98f264c861c215186'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:43:23.388402 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1136/2263] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 24 09:43:23 2022] DELETE /identity/v3/users/da3da89b69264fe98f264c861c215186 => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:43:23.392850 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e922c928-9683-479b-8071-5b39ded9a7a7 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:43:23.393309 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e922c928-9683-479b-8071-5b39ded9a7a7 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:43:23.393427 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e922c928-9683-479b-8071-5b39ded9a7a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:43:23.421552 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-e922c928-9683-479b-8071-5b39ded9a7a7 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:43:23.421552 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-e922c928-9683-479b-8071-5b39ded9a7a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:43:23.481309 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1128/2264] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 09:43:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:43:23.493032 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9349736b-74b7-458c-9b79-a76898eb56dc admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:43:23.494457 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-9349736b-74b7-458c-9b79-a76898eb56dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:43:23.551064 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9349736b-74b7-458c-9b79-a76898eb56dc admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:43:23.554807 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b5035c-6c5c-42d6-8ec7-ed2d376a5d1f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e5260e798404e8bb12b25ca8ea97052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:43:23.555406 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b5035c-6c5c-42d6-8ec7-ed2d376a5d1f service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:43:23.555717 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b5035c-6c5c-42d6-8ec7-ed2d376a5d1f service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:43:23.555992 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b5035c-6c5c-42d6-8ec7-ed2d376a5d1f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:43:23.565322 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b5035c-6c5c-42d6-8ec7-ed2d376a5d1f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:43:23.566725 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b5035c-6c5c-42d6-8ec7-ed2d376a5d1f service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:43:23.575889 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1137/2265] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:43:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:43:23.947755 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e922c928-9683-479b-8071-5b39ded9a7a7 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:43:23.949220 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-e922c928-9683-479b-8071-5b39ded9a7a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:43:23.961106 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e922c928-9683-479b-8071-5b39ded9a7a7 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:43:23.966018 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dbe494-6d6f-40fe-87f3-ea2a8777c1e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:43:23.966692 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dbe494-6d6f-40fe-87f3-ea2a8777c1e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:43:23.966973 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dbe494-6d6f-40fe-87f3-ea2a8777c1e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:43:23.967248 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dbe494-6d6f-40fe-87f3-ea2a8777c1e7 admin admin] PATH_INFO: `/v3/projects/b6168e9f52e6476dad91def3642ac5b7` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:43:23.968963 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6dbe494-6d6f-40fe-87f3-ea2a8777c1e7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b6168e9f52e6476dad91def3642ac5b7)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:43:23.970858 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6dbe494-6d6f-40fe-87f3-ea2a8777c1e7 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:43:24.102736 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-d6dbe494-6d6f-40fe-87f3-ea2a8777c1e7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b6168e9f52e6476dad91def3642ac5b7', 'request_id': 'req-d6dbe494-6d6f-40fe-87f3-ea2a8777c1e7'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:43:24.109278 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-d6dbe494-6d6f-40fe-87f3-ea2a8777c1e7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b6168e9f52e6476dad91def3642ac5b7', 'request_id': 'req-d6dbe494-6d6f-40fe-87f3-ea2a8777c1e7'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:43:24.115121 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-d6dbe494-6d6f-40fe-87f3-ea2a8777c1e7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b6168e9f52e6476dad91def3642ac5b7', 'request_id': 'req-d6dbe494-6d6f-40fe-87f3-ea2a8777c1e7'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:43:24.203665 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-d6dbe494-6d6f-40fe-87f3-ea2a8777c1e7 admin admin] The token cache is being invalidate because project b6168e9f52e6476dad91def3642ac5b7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:43:24.203958 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-d6dbe494-6d6f-40fe-87f3-ea2a8777c1e7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:43:24.204574 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-d6dbe494-6d6f-40fe-87f3-ea2a8777c1e7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:43:24.206384 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1129/2266] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 24 09:43:23 2022] DELETE /identity/v3/projects/b6168e9f52e6476dad91def3642ac5b7 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:43:24.212372 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58545ec6-182d-4f78-be72-7eca23d64ac8 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:43:24.215944 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58545ec6-182d-4f78-be72-7eca23d64ac8 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:43:24.215944 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58545ec6-182d-4f78-be72-7eca23d64ac8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:43:24.231113 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:43:24.253923 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-58545ec6-182d-4f78-be72-7eca23d64ac8 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:43:24.255214 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-58545ec6-182d-4f78-be72-7eca23d64ac8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:43:24.314386 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1138/2267] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 09:43:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:43:24.321947 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d9eedf-493d-4a92-911b-2abce353ca6e None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:43:24.322262 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d9eedf-493d-4a92-911b-2abce353ca6e None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:43:24.322538 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d9eedf-493d-4a92-911b-2abce353ca6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:43:24.343077 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-e5d9eedf-493d-4a92-911b-2abce353ca6e None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:43:24.344964 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-e5d9eedf-493d-4a92-911b-2abce353ca6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:43:24.353225 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1130/2268] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 09:43:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:43:24.357510 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58545ec6-182d-4f78-be72-7eca23d64ac8 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:43:24.366932 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58545ec6-182d-4f78-be72-7eca23d64ac8 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:43:24.371409 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280b064d-ad11-4a3d-9ca8-3622831f0da8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:43:24.372035 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280b064d-ad11-4a3d-9ca8-3622831f0da8 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:43:24.372316 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280b064d-ad11-4a3d-9ca8-3622831f0da8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:43:24.372587 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280b064d-ad11-4a3d-9ca8-3622831f0da8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:43:24.373227 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280b064d-ad11-4a3d-9ca8-3622831f0da8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:43:24.373530 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280b064d-ad11-4a3d-9ca8-3622831f0da8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:43:24.375145 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280b064d-ad11-4a3d-9ca8-3622831f0da8 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:43:24.386516 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1139/2269] 10.222.0.16 () {58 vars in 1153 bytes} [Wed Aug 24 09:43:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:43:24.391570 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d9eedf-493d-4a92-911b-2abce353ca6e None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:43:24.400680 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d9eedf-493d-4a92-911b-2abce353ca6e None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:43:24.405355 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf0e0dc-767a-4166-9d75-e1ac194993ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:43:24.406567 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf0e0dc-767a-4166-9d75-e1ac194993ec admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:43:24.406853 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf0e0dc-767a-4166-9d75-e1ac194993ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:43:24.407120 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf0e0dc-767a-4166-9d75-e1ac194993ec admin admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:43:24.407814 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf0e0dc-767a-4166-9d75-e1ac194993ec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:43:24.409705 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf0e0dc-767a-4166-9d75-e1ac194993ec admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:43:24.516128 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1131/2270] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 24 09:43:24 2022] POST /identity/v3/projects => generated 373 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:43:24.520241 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280b064d-ad11-4a3d-9ca8-3622831f0da8 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:43:24.530292 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280b064d-ad11-4a3d-9ca8-3622831f0da8 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:43:24.585653 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5be0748-623f-431e-8541-42299c7b0c78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:43:24.586250 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5be0748-623f-431e-8541-42299c7b0c78 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:43:24.586531 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5be0748-623f-431e-8541-42299c7b0c78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:43:24.586804 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5be0748-623f-431e-8541-42299c7b0c78 admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:43:24.587420 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5be0748-623f-431e-8541-42299c7b0c78 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:43:24.589383 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5be0748-623f-431e-8541-42299c7b0c78 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:43:24.697287 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1140/2271] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 24 09:43:24 2022] POST /identity/v3/users => generated 287 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:43:24.701368 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf0e0dc-767a-4166-9d75-e1ac194993ec admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:43:24.712814 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf0e0dc-767a-4166-9d75-e1ac194993ec admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:43:24.717488 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fd959e-357b-4593-8c56-fad0922daac8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:43:24.718077 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fd959e-357b-4593-8c56-fad0922daac8 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:43:24.718356 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fd959e-357b-4593-8c56-fad0922daac8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:43:24.718655 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fd959e-357b-4593-8c56-fad0922daac8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:43:24.719284 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fd959e-357b-4593-8c56-fad0922daac8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:43:24.720926 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fd959e-357b-4593-8c56-fad0922daac8 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:43:24.730563 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1132/2272] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 24 09:43:24 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:43:24.734404 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5be0748-623f-431e-8541-42299c7b0c78 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:43:24.744533 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5be0748-623f-431e-8541-42299c7b0c78 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:43:24.749302 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc7abe9-8ac2-4b4a-bd86-5d846b915e41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:43:24.753854 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc7abe9-8ac2-4b4a-bd86-5d846b915e41 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:43:24.753854 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc7abe9-8ac2-4b4a-bd86-5d846b915e41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:43:24.753854 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc7abe9-8ac2-4b4a-bd86-5d846b915e41 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:43:24.753854 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc7abe9-8ac2-4b4a-bd86-5d846b915e41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:43:24.753854 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc7abe9-8ac2-4b4a-bd86-5d846b915e41 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:43:24.763627 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1141/2273] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 24 09:43:24 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:43:24.768245 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fd959e-357b-4593-8c56-fad0922daac8 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:43:24.778026 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fd959e-357b-4593-8c56-fad0922daac8 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:43:24.782569 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cf470a-64ad-4cb0-88d1-91bdc2a4e79f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:43:24.783217 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cf470a-64ad-4cb0-88d1-91bdc2a4e79f admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:43:24.783499 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cf470a-64ad-4cb0-88d1-91bdc2a4e79f admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:43:24.783807 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cf470a-64ad-4cb0-88d1-91bdc2a4e79f admin admin] PATH_INFO: `/v3/projects/0c2cf0b26f6b4d0fa7553727fd7eb053/users/bf2cf8971d4e473dbb8772263381e0ca/roles/136cdf78e00a426997dd4e37faff489b` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:43:24.798554 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cf470a-64ad-4cb0-88d1-91bdc2a4e79f admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c2cf0b26f6b4d0fa7553727fd7eb053, user_id=bf2cf8971d4e473dbb8772263381e0ca, role_id=136cdf78e00a426997dd4e37faff489b)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:43:24.801226 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cf470a-64ad-4cb0-88d1-91bdc2a4e79f admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:43:24.851000 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1133/2274] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 24 09:43:24 2022] PUT /identity/v3/projects/0c2cf0b26f6b4d0fa7553727fd7eb053/users/bf2cf8971d4e473dbb8772263381e0ca/roles/136cdf78e00a426997dd4e37faff489b => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:43:24.861320 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda09fbc-7e47-4b16-804e-9e81cc22981f None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:43:24.861651 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda09fbc-7e47-4b16-804e-9e81cc22981f None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:43:24.861917 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda09fbc-7e47-4b16-804e-9e81cc22981f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:43:24.890495 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-fda09fbc-7e47-4b16-804e-9e81cc22981f None None] MFA Rules not processed for user `bf2cf8971d4e473dbb8772263381e0ca`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:43:24.891739 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-fda09fbc-7e47-4b16-804e-9e81cc22981f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:43:24.940756 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:43:24.956932 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1142/2275] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 09:43:24 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:43:24.967739 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cf470a-64ad-4cb0-88d1-91bdc2a4e79f admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:43:24.969433 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-69cf470a-64ad-4cb0-88d1-91bdc2a4e79f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:43:24.975873 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:43:25.025162 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cf470a-64ad-4cb0-88d1-91bdc2a4e79f admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:43:25.028738 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f5bc51-d9cd-4310-b2de-829e92094bbf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c32b83577c46488906fe409a790b24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:43:25.029354 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f5bc51-d9cd-4310-b2de-829e92094bbf service cinder] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:43:25.029652 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f5bc51-d9cd-4310-b2de-829e92094bbf service cinder] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:43:25.029926 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f5bc51-d9cd-4310-b2de-829e92094bbf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:43:25.038260 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f5bc51-d9cd-4310-b2de-829e92094bbf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:43:25.039717 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f5bc51-d9cd-4310-b2de-829e92094bbf service cinder] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:43:25.049171 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1134/2276] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:43:24 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:43:35.074409 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=4 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:43:35.131650 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=24 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:43:57.227370 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda09fbc-7e47-4b16-804e-9e81cc22981f None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:43:57.228833 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-fda09fbc-7e47-4b16-804e-9e81cc22981f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:43:57.285335 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda09fbc-7e47-4b16-804e-9e81cc22981f None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:43:57.288982 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19470103-7805-487b-950c-46a1abc6ccf5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e5260e798404e8bb12b25ca8ea97052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:43:57.289711 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19470103-7805-487b-950c-46a1abc6ccf5 service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:43:57.289993 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19470103-7805-487b-950c-46a1abc6ccf5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:43:57.290263 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19470103-7805-487b-950c-46a1abc6ccf5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:43:57.292048 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-19470103-7805-487b-950c-46a1abc6ccf5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:43:57.368721 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19470103-7805-487b-950c-46a1abc6ccf5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:43:57.370092 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19470103-7805-487b-950c-46a1abc6ccf5 service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:43:57.381016 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1143/2277] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:43:57 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:43:58.559650 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f5bc51-d9cd-4310-b2de-829e92094bbf service cinder] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:43:58.560855 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-20f5bc51-d9cd-4310-b2de-829e92094bbf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:43:58.573313 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f5bc51-d9cd-4310-b2de-829e92094bbf service cinder] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:43:58.579092 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c479163f-131d-431a-85ca-d3fc59dc507b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e5260e798404e8bb12b25ca8ea97052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:43:58.579824 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c479163f-131d-431a-85ca-d3fc59dc507b service neutron] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:43:58.580132 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c479163f-131d-431a-85ca-d3fc59dc507b service neutron] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:43:58.580350 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c479163f-131d-431a-85ca-d3fc59dc507b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:43:58.582033 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-c479163f-131d-431a-85ca-d3fc59dc507b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:43:58.596595 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c479163f-131d-431a-85ca-d3fc59dc507b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:43:58.598010 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c479163f-131d-431a-85ca-d3fc59dc507b service neutron] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:43:58.608906 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1135/2278] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:43:58 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:44:07.376208 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=54 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:44:08.604174 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=6 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:44:14.800835 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19470103-7805-487b-950c-46a1abc6ccf5 service neutron] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:44:14.809492 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19470103-7805-487b-950c-46a1abc6ccf5 service neutron] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:44:14.813156 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb52df3-c212-450f-b79d-b213336e3da5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e5260e798404e8bb12b25ca8ea97052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:44:14.813758 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb52df3-c212-450f-b79d-b213336e3da5 service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:44:14.814040 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb52df3-c212-450f-b79d-b213336e3da5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:44:14.814316 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb52df3-c212-450f-b79d-b213336e3da5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:44:14.816075 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-0fb52df3-c212-450f-b79d-b213336e3da5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:44:14.825331 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb52df3-c212-450f-b79d-b213336e3da5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:44:14.826726 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb52df3-c212-450f-b79d-b213336e3da5 service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:44:14.834528 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1144/2279] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:44:14 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:44:24.831045 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=6 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:44:56.704935 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c479163f-131d-431a-85ca-d3fc59dc507b service neutron] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:44:56.741254 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c479163f-131d-431a-85ca-d3fc59dc507b service neutron] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:44:56.744846 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f98a31a-366b-46c9-9223-67301022a994 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c32b83577c46488906fe409a790b24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:44:56.745427 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f98a31a-366b-46c9-9223-67301022a994 service cinder] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:44:56.745703 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f98a31a-366b-46c9-9223-67301022a994 service cinder] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:44:56.745971 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f98a31a-366b-46c9-9223-67301022a994 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:44:56.747878 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-1f98a31a-366b-46c9-9223-67301022a994 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:44:56.828850 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f98a31a-366b-46c9-9223-67301022a994 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:44:56.830200 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f98a31a-366b-46c9-9223-67301022a994 service cinder] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:44:56.839705 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1136/2280] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:44:56 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:45:06.835222 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=50 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:45:31.664767 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb52df3-c212-450f-b79d-b213336e3da5 service neutron] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:45:31.683122 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb52df3-c212-450f-b79d-b213336e3da5 service neutron] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:45:31.686754 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04419e44-4cdc-4780-b136-df556e5abd36 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e5260e798404e8bb12b25ca8ea97052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:45:31.687345 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04419e44-4cdc-4780-b136-df556e5abd36 service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:45:31.687683 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04419e44-4cdc-4780-b136-df556e5abd36 service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:45:31.688642 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04419e44-4cdc-4780-b136-df556e5abd36 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:45:31.690533 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-04419e44-4cdc-4780-b136-df556e5abd36 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:45:31.747143 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04419e44-4cdc-4780-b136-df556e5abd36 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:45:31.748597 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04419e44-4cdc-4780-b136-df556e5abd36 service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:45:31.767018 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1145/2281] 10.222.0.16 () {60 vars in 1396 bytes} [Wed Aug 24 09:45:31 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3260 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:45:41.764488 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=37 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:10.896107 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f98a31a-366b-46c9-9223-67301022a994 service cinder] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:10.897567 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-1f98a31a-366b-46c9-9223-67301022a994 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:47:10.922485 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f98a31a-366b-46c9-9223-67301022a994 service cinder] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:10.928139 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4831f284-6f30-4472-8b36-6734f34f7a29 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f055f7dc51d43bda9b315ca67f11d58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:10.928787 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4831f284-6f30-4472-8b36-6734f34f7a29 service nova] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:10.929093 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4831f284-6f30-4472-8b36-6734f34f7a29 service nova] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:10.929516 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4831f284-6f30-4472-8b36-6734f34f7a29 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:10.931354 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-4831f284-6f30-4472-8b36-6734f34f7a29 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:47:10.945224 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4831f284-6f30-4472-8b36-6734f34f7a29 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:10.946971 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4831f284-6f30-4472-8b36-6734f34f7a29 service nova] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:10.959500 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1137/2282] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:47:10 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:47:11.166982 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04419e44-4cdc-4780-b136-df556e5abd36 service neutron] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:11.168647 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-04419e44-4cdc-4780-b136-df556e5abd36 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:47:11.272376 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04419e44-4cdc-4780-b136-df556e5abd36 service neutron] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:11.277891 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf1518e-4e56-48de-9b4e-758f456fea3e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80cada22039d4f11935412ee578acd05', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:11.278850 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf1518e-4e56-48de-9b4e-758f456fea3e service placement] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:11.278978 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf1518e-4e56-48de-9b4e-758f456fea3e service placement] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:11.279165 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf1518e-4e56-48de-9b4e-758f456fea3e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:11.281412 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-cbf1518e-4e56-48de-9b4e-758f456fea3e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:47:11.293849 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf1518e-4e56-48de-9b4e-758f456fea3e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:11.295464 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf1518e-4e56-48de-9b4e-758f456fea3e service placement] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:11.308823 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1146/2283] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:47:11 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:47:20.953754 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=9 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:21.304314 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=41 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:48:01.613826 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4831f284-6f30-4472-8b36-6734f34f7a29 service nova] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:48:01.615237 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-4831f284-6f30-4472-8b36-6734f34f7a29 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:48:01.657306 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4831f284-6f30-4472-8b36-6734f34f7a29 service nova] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:48:01.663308 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8432fe-6ea7-4947-bdb7-9624f56812bb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f055f7dc51d43bda9b315ca67f11d58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:48:01.663931 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8432fe-6ea7-4947-bdb7-9624f56812bb service nova] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:48:01.664213 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8432fe-6ea7-4947-bdb7-9624f56812bb service nova] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:48:01.664486 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8432fe-6ea7-4947-bdb7-9624f56812bb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:48:01.666186 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-8e8432fe-6ea7-4947-bdb7-9624f56812bb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:48:01.743932 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8432fe-6ea7-4947-bdb7-9624f56812bb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:48:01.745550 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8432fe-6ea7-4947-bdb7-9624f56812bb service nova] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:48:01.754428 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1138/2284] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:48:01 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 09:48:11.750269 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=49 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:48:57.970103 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf1518e-4e56-48de-9b4e-758f456fea3e service placement] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:48:57.988239 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf1518e-4e56-48de-9b4e-758f456fea3e service placement] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:48:57.992327 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca4152a-eef2-49e5-afa8-cda979430088 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e5260e798404e8bb12b25ca8ea97052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:48:57.992913 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca4152a-eef2-49e5-afa8-cda979430088 service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:48:57.993192 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca4152a-eef2-49e5-afa8-cda979430088 service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:48:57.993463 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca4152a-eef2-49e5-afa8-cda979430088 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:48:58.003445 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca4152a-eef2-49e5-afa8-cda979430088 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:48:58.004874 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca4152a-eef2-49e5-afa8-cda979430088 service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:48:58.016055 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1147/2285] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:48:57 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:48:59.286292 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8432fe-6ea7-4947-bdb7-9624f56812bb service nova] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:48:59.295874 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8432fe-6ea7-4947-bdb7-9624f56812bb service nova] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:48:59.299394 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cd6675-8042-49db-9c32-f858fa1920bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e5260e798404e8bb12b25ca8ea97052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:48:59.300024 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cd6675-8042-49db-9c32-f858fa1920bc service neutron] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:48:59.300302 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cd6675-8042-49db-9c32-f858fa1920bc service neutron] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:48:59.300569 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cd6675-8042-49db-9c32-f858fa1920bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:48:59.311330 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cd6675-8042-49db-9c32-f858fa1920bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:48:59.312769 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cd6675-8042-49db-9c32-f858fa1920bc service neutron] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:48:59.323275 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1139/2286] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:48:59 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:49:08.011379 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=10 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:49:09.318594 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=6 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:49:19.827320 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca4152a-eef2-49e5-afa8-cda979430088 service neutron] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:19.836134 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca4152a-eef2-49e5-afa8-cda979430088 service neutron] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:19.839701 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843b4e47-ef53-465d-bc8d-7f9459465c94 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e5260e798404e8bb12b25ca8ea97052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:19.840284 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843b4e47-ef53-465d-bc8d-7f9459465c94 service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:19.840561 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843b4e47-ef53-465d-bc8d-7f9459465c94 service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:19.840831 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843b4e47-ef53-465d-bc8d-7f9459465c94 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:19.847803 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843b4e47-ef53-465d-bc8d-7f9459465c94 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:49:19.849715 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843b4e47-ef53-465d-bc8d-7f9459465c94 service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:49:19.857494 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1148/2287] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:49:19 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 31 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:49:29.854039 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=6 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:49:30.433720 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cd6675-8042-49db-9c32-f858fa1920bc service neutron] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:30.444009 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cd6675-8042-49db-9c32-f858fa1920bc service neutron] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:30.447535 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3570ecdf-9e3c-45b7-8981-274e0daaba86 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c32b83577c46488906fe409a790b24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:30.448157 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3570ecdf-9e3c-45b7-8981-274e0daaba86 service cinder] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:30.448435 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3570ecdf-9e3c-45b7-8981-274e0daaba86 service cinder] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:30.448705 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3570ecdf-9e3c-45b7-8981-274e0daaba86 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:30.450445 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-3570ecdf-9e3c-45b7-8981-274e0daaba86 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:49:30.528692 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3570ecdf-9e3c-45b7-8981-274e0daaba86 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:49:30.530110 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3570ecdf-9e3c-45b7-8981-274e0daaba86 service cinder] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:49:30.547665 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1140/2288] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:49:30 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:49:30.591061 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843b4e47-ef53-465d-bc8d-7f9459465c94 service neutron] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:30.600199 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843b4e47-ef53-465d-bc8d-7f9459465c94 service neutron] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:30.603962 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76aea44b-791f-45ac-ab64-5607de3e69c7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c32b83577c46488906fe409a790b24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:30.604555 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76aea44b-791f-45ac-ab64-5607de3e69c7 service cinder] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:30.604835 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76aea44b-791f-45ac-ab64-5607de3e69c7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:30.605100 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76aea44b-791f-45ac-ab64-5607de3e69c7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:30.606822 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-76aea44b-791f-45ac-ab64-5607de3e69c7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:49:30.684614 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76aea44b-791f-45ac-ab64-5607de3e69c7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:49:30.686017 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76aea44b-791f-45ac-ab64-5607de3e69c7 service cinder] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:49:30.704094 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1149/2289] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:49:30 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:49:30.747042 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3570ecdf-9e3c-45b7-8981-274e0daaba86 service cinder] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:30.755426 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3570ecdf-9e3c-45b7-8981-274e0daaba86 service cinder] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:30.758976 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173c0b90-5aeb-453c-af65-f7c81c0f79eb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c32b83577c46488906fe409a790b24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:30.759564 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173c0b90-5aeb-453c-af65-f7c81c0f79eb service cinder] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:30.759868 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173c0b90-5aeb-453c-af65-f7c81c0f79eb service cinder] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:30.760138 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173c0b90-5aeb-453c-af65-f7c81c0f79eb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:30.768037 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173c0b90-5aeb-453c-af65-f7c81c0f79eb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:49:30.769429 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173c0b90-5aeb-453c-af65-f7c81c0f79eb service cinder] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:49:30.778672 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1141/2290] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:49:30 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 32 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:49:32.443061 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76aea44b-791f-45ac-ab64-5607de3e69c7 service cinder] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:32.530681 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76aea44b-791f-45ac-ab64-5607de3e69c7 service cinder] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:32.535907 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a437e66-2fd8-4971-b94e-9f78f196eec8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:32.536599 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a437e66-2fd8-4971-b94e-9f78f196eec8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:32.536883 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a437e66-2fd8-4971-b94e-9f78f196eec8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:32.537159 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a437e66-2fd8-4971-b94e-9f78f196eec8 admin admin] PATH_INFO: `/v3/users/bf2cf8971d4e473dbb8772263381e0ca` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:32.538326 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a437e66-2fd8-4971-b94e-9f78f196eec8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf2cf8971d4e473dbb8772263381e0ca)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:49:32.540293 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a437e66-2fd8-4971-b94e-9f78f196eec8 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:49:32.694814 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-9a437e66-2fd8-4971-b94e-9f78f196eec8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf2cf8971d4e473dbb8772263381e0ca'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:49:32.695381 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-9a437e66-2fd8-4971-b94e-9f78f196eec8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf2cf8971d4e473dbb8772263381e0ca'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:49:32.754815 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-9a437e66-2fd8-4971-b94e-9f78f196eec8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf2cf8971d4e473dbb8772263381e0ca'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:49:32.755427 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-9a437e66-2fd8-4971-b94e-9f78f196eec8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf2cf8971d4e473dbb8772263381e0ca'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:49:32.756721 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:49:32.762796 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-9a437e66-2fd8-4971-b94e-9f78f196eec8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf2cf8971d4e473dbb8772263381e0ca'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:49:32.780208 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1150/2291] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 24 09:49:32 2022] DELETE /identity/v3/users/bf2cf8971d4e473dbb8772263381e0ca => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:49:32.784706 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79287c98-a84e-4a31-b12d-a565bd93fe66 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:32.785320 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79287c98-a84e-4a31-b12d-a565bd93fe66 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:32.786021 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79287c98-a84e-4a31-b12d-a565bd93fe66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:32.816989 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-79287c98-a84e-4a31-b12d-a565bd93fe66 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:49:32.818204 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-79287c98-a84e-4a31-b12d-a565bd93fe66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:49:32.876955 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1142/2292] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 09:49:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:49:32.886963 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a437e66-2fd8-4971-b94e-9f78f196eec8 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:32.888419 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-9a437e66-2fd8-4971-b94e-9f78f196eec8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:49:32.955948 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a437e66-2fd8-4971-b94e-9f78f196eec8 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:32.960320 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169d627d-042f-4b31-8ddc-f286e37a90f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e5260e798404e8bb12b25ca8ea97052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:32.960921 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169d627d-042f-4b31-8ddc-f286e37a90f3 service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:32.961201 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169d627d-042f-4b31-8ddc-f286e37a90f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:32.961483 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169d627d-042f-4b31-8ddc-f286e37a90f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:32.970201 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169d627d-042f-4b31-8ddc-f286e37a90f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:49:32.971644 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169d627d-042f-4b31-8ddc-f286e37a90f3 service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:49:32.981515 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1151/2293] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:49:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:49:33.420526 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79287c98-a84e-4a31-b12d-a565bd93fe66 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:33.428444 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-79287c98-a84e-4a31-b12d-a565bd93fe66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:49:33.434172 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79287c98-a84e-4a31-b12d-a565bd93fe66 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:33.440375 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bf03ea-3bdb-4627-8581-41ea808cb311 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:33.441022 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bf03ea-3bdb-4627-8581-41ea808cb311 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:33.441299 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bf03ea-3bdb-4627-8581-41ea808cb311 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:33.442176 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bf03ea-3bdb-4627-8581-41ea808cb311 admin admin] PATH_INFO: `/v3/projects/0c2cf0b26f6b4d0fa7553727fd7eb053` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:33.443323 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bf03ea-3bdb-4627-8581-41ea808cb311 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0c2cf0b26f6b4d0fa7553727fd7eb053)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:49:33.445762 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bf03ea-3bdb-4627-8581-41ea808cb311 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:49:33.564064 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:49:33.571023 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-a9bf03ea-3bdb-4627-8581-41ea808cb311 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0c2cf0b26f6b4d0fa7553727fd7eb053', 'request_id': 'req-a9bf03ea-3bdb-4627-8581-41ea808cb311'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:49:33.577458 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-a9bf03ea-3bdb-4627-8581-41ea808cb311 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0c2cf0b26f6b4d0fa7553727fd7eb053', 'request_id': 'req-a9bf03ea-3bdb-4627-8581-41ea808cb311'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:49:33.583503 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-a9bf03ea-3bdb-4627-8581-41ea808cb311 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0c2cf0b26f6b4d0fa7553727fd7eb053', 'request_id': 'req-a9bf03ea-3bdb-4627-8581-41ea808cb311'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:49:33.653237 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-a9bf03ea-3bdb-4627-8581-41ea808cb311 admin admin] The token cache is being invalidate because project 0c2cf0b26f6b4d0fa7553727fd7eb053 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:49:33.653534 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-a9bf03ea-3bdb-4627-8581-41ea808cb311 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:49:33.654164 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-a9bf03ea-3bdb-4627-8581-41ea808cb311 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:49:33.655725 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1143/2294] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 24 09:49:33 2022] DELETE /identity/v3/projects/0c2cf0b26f6b4d0fa7553727fd7eb053 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:49:42.977313 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=42,keystone:DELETE=2 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:49:43.784767 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=9,keystone:DELETE=4,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:31.091501 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169d627d-042f-4b31-8ddc-f286e37a90f3 service neutron] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:31.092929 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-169d627d-042f-4b31-8ddc-f286e37a90f3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:51:31.158806 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169d627d-042f-4b31-8ddc-f286e37a90f3 service neutron] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:31.207331 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa303ef-8544-411b-9377-f57b813d7254 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c32b83577c46488906fe409a790b24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:31.208224 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa303ef-8544-411b-9377-f57b813d7254 service cinder] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:31.209210 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa303ef-8544-411b-9377-f57b813d7254 service cinder] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:31.209443 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa303ef-8544-411b-9377-f57b813d7254 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:31.211305 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-dfa303ef-8544-411b-9377-f57b813d7254 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:51:31.292242 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa303ef-8544-411b-9377-f57b813d7254 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:31.293701 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa303ef-8544-411b-9377-f57b813d7254 service cinder] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:31.313946 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1152/2295] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:51:31 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:51:31.356748 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bf03ea-3bdb-4627-8581-41ea808cb311 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:31.358229 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-a9bf03ea-3bdb-4627-8581-41ea808cb311 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:51:31.369993 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bf03ea-3bdb-4627-8581-41ea808cb311 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:31.373506 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c735202-2564-47c2-8566-a950d33539ff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c32b83577c46488906fe409a790b24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:31.374098 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c735202-2564-47c2-8566-a950d33539ff service cinder] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:31.374373 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c735202-2564-47c2-8566-a950d33539ff service cinder] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:31.374639 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c735202-2564-47c2-8566-a950d33539ff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:31.376323 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-4c735202-2564-47c2-8566-a950d33539ff service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:51:31.455772 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c735202-2564-47c2-8566-a950d33539ff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:31.457207 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c735202-2564-47c2-8566-a950d33539ff service cinder] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:31.467124 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1144/2296] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:51:31 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:51:41.311112 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=86 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:41.462802 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=37 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:52:18.163331 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa303ef-8544-411b-9377-f57b813d7254 service cinder] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:52:18.164779 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-dfa303ef-8544-411b-9377-f57b813d7254 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:52:18.232054 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa303ef-8544-411b-9377-f57b813d7254 service cinder] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:52:18.237027 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49330379-4a53-40cd-976f-1e854d5f8566 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80cada22039d4f11935412ee578acd05', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:52:18.238129 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49330379-4a53-40cd-976f-1e854d5f8566 service placement] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:52:18.238406 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49330379-4a53-40cd-976f-1e854d5f8566 service placement] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:52:18.238674 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49330379-4a53-40cd-976f-1e854d5f8566 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:52:18.240412 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-49330379-4a53-40cd-976f-1e854d5f8566 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:52:18.251377 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49330379-4a53-40cd-976f-1e854d5f8566 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:52:18.252997 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49330379-4a53-40cd-976f-1e854d5f8566 service placement] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:52:18.263140 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1153/2297] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:52:18 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:52:28.259155 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=32 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:53:01.465234 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c735202-2564-47c2-8566-a950d33539ff service cinder] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:53:01.469079 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-4c735202-2564-47c2-8566-a950d33539ff service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:53:01.521960 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c735202-2564-47c2-8566-a950d33539ff service cinder] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:53:01.525629 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed05ffc2-980f-446f-9201-1765df78ba7d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e5260e798404e8bb12b25ca8ea97052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:53:01.526222 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed05ffc2-980f-446f-9201-1765df78ba7d service neutron] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:53:01.526499 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed05ffc2-980f-446f-9201-1765df78ba7d service neutron] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:53:01.526769 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed05ffc2-980f-446f-9201-1765df78ba7d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:53:01.528445 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-ed05ffc2-980f-446f-9201-1765df78ba7d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:53:01.587976 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed05ffc2-980f-446f-9201-1765df78ba7d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:53:01.589333 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed05ffc2-980f-446f-9201-1765df78ba7d service neutron] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:53:01.599177 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1145/2298] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:53:01 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:53:02.960948 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ec1976-07aa-4e4b-9f1a-4e62340e57a9 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:53:02.960948 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ec1976-07aa-4e4b-9f1a-4e62340e57a9 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:53:02.960948 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ec1976-07aa-4e4b-9f1a-4e62340e57a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:53:02.960948 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-10ec1976-07aa-4e4b-9f1a-4e62340e57a9 None None] MFA Rules not processed for user `def04335e57e48409a6943fa231d47c3`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:53:02.960948 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-10ec1976-07aa-4e4b-9f1a-4e62340e57a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:53:02.960948 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1154/2299] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 09:53:02 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:53:03.243444 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed05ffc2-980f-446f-9201-1765df78ba7d service neutron] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:53:03.244896 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-ed05ffc2-980f-446f-9201-1765df78ba7d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:53:03.255964 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed05ffc2-980f-446f-9201-1765df78ba7d service neutron] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:53:03.259523 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a70517-9c51-44b2-8562-bad06b42c5eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e5260e798404e8bb12b25ca8ea97052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:53:03.260143 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a70517-9c51-44b2-8562-bad06b42c5eb service neutron] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:53:03.260419 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a70517-9c51-44b2-8562-bad06b42c5eb service neutron] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:53:03.260688 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a70517-9c51-44b2-8562-bad06b42c5eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:53:03.262327 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-b9a70517-9c51-44b2-8562-bad06b42c5eb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:53:03.271323 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a70517-9c51-44b2-8562-bad06b42c5eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:53:03.272858 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a70517-9c51-44b2-8562-bad06b42c5eb service neutron] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:53:03.280861 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1146/2300] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:53:03 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:53:03.683440 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ec1976-07aa-4e4b-9f1a-4e62340e57a9 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:53:03.684867 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-10ec1976-07aa-4e4b-9f1a-4e62340e57a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:53:03.700194 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ec1976-07aa-4e4b-9f1a-4e62340e57a9 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:53:03.705136 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e83ec3-794d-4807-84d0-1c361f0167ba service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80cada22039d4f11935412ee578acd05', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:53:03.705841 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e83ec3-794d-4807-84d0-1c361f0167ba service placement] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:53:03.706070 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e83ec3-794d-4807-84d0-1c361f0167ba service placement] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:53:03.707004 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e83ec3-794d-4807-84d0-1c361f0167ba service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:53:03.709009 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-d0e83ec3-794d-4807-84d0-1c361f0167ba service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:53:03.724076 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e83ec3-794d-4807-84d0-1c361f0167ba service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:53:03.724076 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e83ec3-794d-4807-84d0-1c361f0167ba service placement] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:53:03.734067 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1155/2301] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:53:03 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:53:07.193571 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a70517-9c51-44b2-8562-bad06b42c5eb service neutron] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:53:07.195111 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-b9a70517-9c51-44b2-8562-bad06b42c5eb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:53:07.271629 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a70517-9c51-44b2-8562-bad06b42c5eb service neutron] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:53:07.277781 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528bb9db-4c7e-49b3-a80d-cc242d083462 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f055f7dc51d43bda9b315ca67f11d58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:53:07.280924 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528bb9db-4c7e-49b3-a80d-cc242d083462 service nova] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:53:07.280924 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528bb9db-4c7e-49b3-a80d-cc242d083462 service nova] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:53:07.280924 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528bb9db-4c7e-49b3-a80d-cc242d083462 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:53:07.280924 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-528bb9db-4c7e-49b3-a80d-cc242d083462 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:53:07.294458 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528bb9db-4c7e-49b3-a80d-cc242d083462 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:53:07.296082 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528bb9db-4c7e-49b3-a80d-cc242d083462 service nova] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:53:07.308450 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1147/2302] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:53:07 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:53:13.729827 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=16 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:53:14.219263 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e83ec3-794d-4807-84d0-1c361f0167ba service placement] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:53:14.234117 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e83ec3-794d-4807-84d0-1c361f0167ba service placement] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:53:14.240356 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd4feb2-7afe-4a83-bab9-f3918d20401a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f055f7dc51d43bda9b315ca67f11d58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:53:14.241083 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd4feb2-7afe-4a83-bab9-f3918d20401a service nova] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:53:14.241364 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd4feb2-7afe-4a83-bab9-f3918d20401a service nova] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:53:14.241636 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd4feb2-7afe-4a83-bab9-f3918d20401a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:53:14.243356 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-6cd4feb2-7afe-4a83-bab9-f3918d20401a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:53:14.257080 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd4feb2-7afe-4a83-bab9-f3918d20401a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:53:14.258701 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd4feb2-7afe-4a83-bab9-f3918d20401a service nova] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:53:14.269597 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1156/2303] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:53:14 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:53:17.303638 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=88 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:53:24.264922 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=6 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:53:40.232886 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528bb9db-4c7e-49b3-a80d-cc242d083462 service nova] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:53:40.242010 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528bb9db-4c7e-49b3-a80d-cc242d083462 service nova] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:53:40.245519 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632ae369-6b1c-437c-9e0d-f4f5805060c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e5260e798404e8bb12b25ca8ea97052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:53:40.246109 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632ae369-6b1c-437c-9e0d-f4f5805060c4 service neutron] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:53:40.246388 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632ae369-6b1c-437c-9e0d-f4f5805060c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:53:40.246654 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632ae369-6b1c-437c-9e0d-f4f5805060c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:53:40.248366 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-632ae369-6b1c-437c-9e0d-f4f5805060c4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:53:40.309899 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632ae369-6b1c-437c-9e0d-f4f5805060c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:53:40.311342 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632ae369-6b1c-437c-9e0d-f4f5805060c4 service neutron] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:53:40.321219 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1148/2304] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:53:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:53:43.677825 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd4feb2-7afe-4a83-bab9-f3918d20401a service nova] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:53:43.689499 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd4feb2-7afe-4a83-bab9-f3918d20401a service nova] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:53:43.695645 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fe6066-1519-4d69-bbb5-3214ee42f0fb service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80cada22039d4f11935412ee578acd05', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:53:43.699433 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fe6066-1519-4d69-bbb5-3214ee42f0fb service placement] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:53:43.699433 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fe6066-1519-4d69-bbb5-3214ee42f0fb service placement] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:53:43.699433 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fe6066-1519-4d69-bbb5-3214ee42f0fb service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:53:43.699433 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-65fe6066-1519-4d69-bbb5-3214ee42f0fb service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:53:43.711143 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fe6066-1519-4d69-bbb5-3214ee42f0fb service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:53:43.712765 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fe6066-1519-4d69-bbb5-3214ee42f0fb service placement] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:53:43.723407 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1157/2305] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:53:43 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:53:43.822534 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632ae369-6b1c-437c-9e0d-f4f5805060c4 service neutron] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:53:43.822534 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-632ae369-6b1c-437c-9e0d-f4f5805060c4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:53:43.834116 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632ae369-6b1c-437c-9e0d-f4f5805060c4 service neutron] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:53:43.838825 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b2a306-f67c-489b-a22c-1dba503febcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:53:43.839504 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b2a306-f67c-489b-a22c-1dba503febcb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:53:43.839789 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b2a306-f67c-489b-a22c-1dba503febcb admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:53:43.840067 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b2a306-f67c-489b-a22c-1dba503febcb admin admin] PATH_INFO: `/v3/users/0aff36cf71984238b8a9b998043ec7be` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:53:43.841182 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b2a306-f67c-489b-a22c-1dba503febcb admin admin] RBAC: Authorizing `identity:delete_user(user_id=0aff36cf71984238b8a9b998043ec7be)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:53:43.843110 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b2a306-f67c-489b-a22c-1dba503febcb admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:53:44.019967 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-b7b2a306-f67c-489b-a22c-1dba503febcb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0aff36cf71984238b8a9b998043ec7be'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:53:44.020607 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-b7b2a306-f67c-489b-a22c-1dba503febcb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0aff36cf71984238b8a9b998043ec7be'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:53:44.032683 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-b7b2a306-f67c-489b-a22c-1dba503febcb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0aff36cf71984238b8a9b998043ec7be'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:53:44.039923 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-b7b2a306-f67c-489b-a22c-1dba503febcb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0aff36cf71984238b8a9b998043ec7be'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:53:44.109940 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-b7b2a306-f67c-489b-a22c-1dba503febcb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0aff36cf71984238b8a9b998043ec7be'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:53:44.116298 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1149/2306] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 24 09:53:43 2022] DELETE /identity/v3/users/0aff36cf71984238b8a9b998043ec7be => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:53:44.366886 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fe6066-1519-4d69-bbb5-3214ee42f0fb service placement] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:53:44.381696 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fe6066-1519-4d69-bbb5-3214ee42f0fb service placement] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:53:44.439268 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec14bb6b-d68c-45e3-919b-66f2cdc08b83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:53:44.439928 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec14bb6b-d68c-45e3-919b-66f2cdc08b83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:53:44.440207 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec14bb6b-d68c-45e3-919b-66f2cdc08b83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:53:44.440475 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec14bb6b-d68c-45e3-919b-66f2cdc08b83 admin admin] PATH_INFO: `/v3/projects/7a0a723d97d24385875a287252f57f02` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:53:44.441650 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec14bb6b-d68c-45e3-919b-66f2cdc08b83 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7a0a723d97d24385875a287252f57f02)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:53:44.444172 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec14bb6b-d68c-45e3-919b-66f2cdc08b83 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:53:44.536934 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-ec14bb6b-d68c-45e3-919b-66f2cdc08b83 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a0a723d97d24385875a287252f57f02', 'request_id': 'req-ec14bb6b-d68c-45e3-919b-66f2cdc08b83'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:53:44.542821 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-ec14bb6b-d68c-45e3-919b-66f2cdc08b83 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a0a723d97d24385875a287252f57f02', 'request_id': 'req-ec14bb6b-d68c-45e3-919b-66f2cdc08b83'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:53:44.548309 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-ec14bb6b-d68c-45e3-919b-66f2cdc08b83 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a0a723d97d24385875a287252f57f02', 'request_id': 'req-ec14bb6b-d68c-45e3-919b-66f2cdc08b83'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:53:44.628640 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-ec14bb6b-d68c-45e3-919b-66f2cdc08b83 admin admin] The token cache is being invalidate because project 7a0a723d97d24385875a287252f57f02 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81602) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:53:44.629021 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-ec14bb6b-d68c-45e3-919b-66f2cdc08b83 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:53:44.630243 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-ec14bb6b-d68c-45e3-919b-66f2cdc08b83 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:53:44.631760 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1158/2307] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 24 09:53:44 2022] DELETE /identity/v3/projects/7a0a723d97d24385875a287252f57f02 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:53:44.637559 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6454dc0d-d2d3-4350-9882-8a9cc97530ae None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:53:44.637853 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6454dc0d-d2d3-4350-9882-8a9cc97530ae None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:53:44.638128 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6454dc0d-d2d3-4350-9882-8a9cc97530ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:53:44.695426 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-6454dc0d-d2d3-4350-9882-8a9cc97530ae None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:53:44.696674 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-6454dc0d-d2d3-4350-9882-8a9cc97530ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:53:44.725765 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:53:44.754669 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1150/2308] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 09:53:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:53:44.762594 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab86157b-1ff4-4267-b636-4994c87ba750 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:53:44.763063 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab86157b-1ff4-4267-b636-4994c87ba750 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:53:44.763167 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab86157b-1ff4-4267-b636-4994c87ba750 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:53:44.782322 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-ab86157b-1ff4-4267-b636-4994c87ba750 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:53:44.783521 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-ab86157b-1ff4-4267-b636-4994c87ba750 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:53:44.792063 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1159/2309] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 09:53:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:53:44.796291 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6454dc0d-d2d3-4350-9882-8a9cc97530ae None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:53:44.806593 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6454dc0d-d2d3-4350-9882-8a9cc97530ae None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:53:44.811718 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7647adf-86d7-484f-90de-f405e1dfebcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:53:44.812320 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7647adf-86d7-484f-90de-f405e1dfebcb admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:53:44.812599 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7647adf-86d7-484f-90de-f405e1dfebcb admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:53:44.812873 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7647adf-86d7-484f-90de-f405e1dfebcb admin admin] PATH_INFO: `/v3/domains` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:53:44.813522 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7647adf-86d7-484f-90de-f405e1dfebcb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:53:44.813830 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7647adf-86d7-484f-90de-f405e1dfebcb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:53:44.815430 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7647adf-86d7-484f-90de-f405e1dfebcb admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:53:44.826740 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1151/2310] 10.222.0.16 () {58 vars in 1153 bytes} [Wed Aug 24 09:53:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:53:44.830777 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab86157b-1ff4-4267-b636-4994c87ba750 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:53:44.840177 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab86157b-1ff4-4267-b636-4994c87ba750 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:53:44.844812 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25904dc-6e42-44ba-b5af-ad633b01ffec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:53:44.845414 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25904dc-6e42-44ba-b5af-ad633b01ffec admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:53:44.845693 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25904dc-6e42-44ba-b5af-ad633b01ffec admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:53:44.845965 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25904dc-6e42-44ba-b5af-ad633b01ffec admin admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:53:44.846578 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25904dc-6e42-44ba-b5af-ad633b01ffec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:53:44.849024 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25904dc-6e42-44ba-b5af-ad633b01ffec admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:53:44.939473 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1160/2311] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 24 09:53:44 2022] POST /identity/v3/projects => generated 373 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:53:44.943560 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7647adf-86d7-484f-90de-f405e1dfebcb admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:53:44.953387 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7647adf-86d7-484f-90de-f405e1dfebcb admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:53:45.008383 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bee6fc-4aac-431d-a474-610cc5a114bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:53:45.008984 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bee6fc-4aac-431d-a474-610cc5a114bf admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:53:45.009267 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bee6fc-4aac-431d-a474-610cc5a114bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:53:45.009538 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bee6fc-4aac-431d-a474-610cc5a114bf admin admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:53:45.010164 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bee6fc-4aac-431d-a474-610cc5a114bf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:53:45.012118 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bee6fc-4aac-431d-a474-610cc5a114bf admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:53:45.109851 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1152/2312] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 24 09:53:44 2022] POST /identity/v3/users => generated 287 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:53:45.113880 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25904dc-6e42-44ba-b5af-ad633b01ffec admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:53:45.124552 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25904dc-6e42-44ba-b5af-ad633b01ffec admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:53:45.129113 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5404bd5a-e5da-4f7b-bb11-017fa565fc92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:53:45.130362 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5404bd5a-e5da-4f7b-bb11-017fa565fc92 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:53:45.131862 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5404bd5a-e5da-4f7b-bb11-017fa565fc92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:53:45.131862 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5404bd5a-e5da-4f7b-bb11-017fa565fc92 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:53:45.131862 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5404bd5a-e5da-4f7b-bb11-017fa565fc92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:53:45.133171 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5404bd5a-e5da-4f7b-bb11-017fa565fc92 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:53:45.142914 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1161/2313] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 24 09:53:45 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:53:45.147208 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bee6fc-4aac-431d-a474-610cc5a114bf admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:53:45.156956 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bee6fc-4aac-431d-a474-610cc5a114bf admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:53:45.162393 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cf1958-8e04-4821-865a-6b41a9b48a30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:53:45.162989 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cf1958-8e04-4821-865a-6b41a9b48a30 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:53:45.163268 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cf1958-8e04-4821-865a-6b41a9b48a30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:53:45.163540 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cf1958-8e04-4821-865a-6b41a9b48a30 admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:53:45.164211 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cf1958-8e04-4821-865a-6b41a9b48a30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:53:45.165788 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cf1958-8e04-4821-865a-6b41a9b48a30 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:53:45.176266 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1153/2314] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 24 09:53:45 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:53:45.180458 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5404bd5a-e5da-4f7b-bb11-017fa565fc92 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:53:45.190150 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5404bd5a-e5da-4f7b-bb11-017fa565fc92 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:53:45.195086 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115b4ae0-ca3a-40ab-86b9-c92d8bc7de74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:53:45.195762 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115b4ae0-ca3a-40ab-86b9-c92d8bc7de74 admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:53:45.196042 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115b4ae0-ca3a-40ab-86b9-c92d8bc7de74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:53:45.196319 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115b4ae0-ca3a-40ab-86b9-c92d8bc7de74 admin admin] PATH_INFO: `/v3/projects/1ba34edd65e546edb5d360f0bcec102c/users/48b02582c40e49ee9f1f3b8ced324d16/roles/136cdf78e00a426997dd4e37faff489b` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:53:45.211445 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115b4ae0-ca3a-40ab-86b9-c92d8bc7de74 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ba34edd65e546edb5d360f0bcec102c, user_id=48b02582c40e49ee9f1f3b8ced324d16, role_id=136cdf78e00a426997dd4e37faff489b)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:53:45.214098 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115b4ae0-ca3a-40ab-86b9-c92d8bc7de74 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:53:45.304497 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1162/2315] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 24 09:53:45 2022] PUT /identity/v3/projects/1ba34edd65e546edb5d360f0bcec102c/users/48b02582c40e49ee9f1f3b8ced324d16/roles/136cdf78e00a426997dd4e37faff489b => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:53:45.315292 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8af72c-929b-469f-af58-78d08555d048 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:53:45.315597 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8af72c-929b-469f-af58-78d08555d048 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:53:45.315888 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8af72c-929b-469f-af58-78d08555d048 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:53:45.344595 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-1c8af72c-929b-469f-af58-78d08555d048 None None] MFA Rules not processed for user `48b02582c40e49ee9f1f3b8ced324d16`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:53:45.345812 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-1c8af72c-929b-469f-af58-78d08555d048 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:53:45.408742 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1154/2316] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 09:53:45 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:53:45.413777 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115b4ae0-ca3a-40ab-86b9-c92d8bc7de74 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:53:45.419381 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:53:45.474777 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115b4ae0-ca3a-40ab-86b9-c92d8bc7de74 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:53:45.479449 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c0d81b-243a-45ae-9179-11b0617779da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:53:45.480091 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c0d81b-243a-45ae-9179-11b0617779da admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:53:45.480374 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c0d81b-243a-45ae-9179-11b0617779da admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:53:45.480649 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c0d81b-243a-45ae-9179-11b0617779da admin admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:53:45.481280 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c0d81b-243a-45ae-9179-11b0617779da admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:53:45.483293 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c0d81b-243a-45ae-9179-11b0617779da admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:53:45.596705 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1163/2317] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 24 09:53:45 2022] POST /identity/v3/projects => generated 373 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:53:45.600883 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8af72c-929b-469f-af58-78d08555d048 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:53:45.611563 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8af72c-929b-469f-af58-78d08555d048 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:53:45.640491 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:53:45.667909 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9797fc0-e169-469c-a38e-687344e2ac4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:53:45.668526 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9797fc0-e169-469c-a38e-687344e2ac4f admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:53:45.668999 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9797fc0-e169-469c-a38e-687344e2ac4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:53:45.669444 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9797fc0-e169-469c-a38e-687344e2ac4f admin admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:53:45.670484 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9797fc0-e169-469c-a38e-687344e2ac4f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:53:45.672671 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9797fc0-e169-469c-a38e-687344e2ac4f admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:53:45.778342 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1155/2318] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 24 09:53:45 2022] POST /identity/v3/users => generated 293 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:53:45.782317 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c0d81b-243a-45ae-9179-11b0617779da admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:53:45.792729 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c0d81b-243a-45ae-9179-11b0617779da admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:53:45.797401 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7836061-5944-40d1-b80d-a12c6718b5d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:53:45.797998 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7836061-5944-40d1-b80d-a12c6718b5d5 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:53:45.798278 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7836061-5944-40d1-b80d-a12c6718b5d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:53:45.798551 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7836061-5944-40d1-b80d-a12c6718b5d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:53:45.799193 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7836061-5944-40d1-b80d-a12c6718b5d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:53:45.800817 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7836061-5944-40d1-b80d-a12c6718b5d5 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:53:45.812604 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1164/2319] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 24 09:53:45 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:53:45.816656 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9797fc0-e169-469c-a38e-687344e2ac4f admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:53:45.827081 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9797fc0-e169-469c-a38e-687344e2ac4f admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:53:45.831653 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9662cce-2f09-435f-923d-966d52e1207f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:53:45.832557 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9662cce-2f09-435f-923d-966d52e1207f admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:53:45.832902 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9662cce-2f09-435f-923d-966d52e1207f admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:53:45.833743 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9662cce-2f09-435f-923d-966d52e1207f admin admin] PATH_INFO: `/v3/projects/a192611a0d104261ae90599349375c08/users/8182ed6e1e904a5b903f5d3ae8211b4e/roles/15f634d4a2d247ddab90c458e5e1c141` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:53:45.848534 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9662cce-2f09-435f-923d-966d52e1207f admin admin] RBAC: Authorizing `identity:create_grant(project_id=a192611a0d104261ae90599349375c08, user_id=8182ed6e1e904a5b903f5d3ae8211b4e, role_id=15f634d4a2d247ddab90c458e5e1c141)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:53:45.851169 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9662cce-2f09-435f-923d-966d52e1207f admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:53:45.921241 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1156/2320] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 24 09:53:45 2022] PUT /identity/v3/projects/a192611a0d104261ae90599349375c08/users/8182ed6e1e904a5b903f5d3ae8211b4e/roles/15f634d4a2d247ddab90c458e5e1c141 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:53:45.925262 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7836061-5944-40d1-b80d-a12c6718b5d5 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:53:45.935801 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7836061-5944-40d1-b80d-a12c6718b5d5 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:53:45.991574 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b7cdb2-0c1d-4900-8928-439dd480a554 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:53:45.992197 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b7cdb2-0c1d-4900-8928-439dd480a554 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:53:45.992479 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b7cdb2-0c1d-4900-8928-439dd480a554 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:53:45.992755 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b7cdb2-0c1d-4900-8928-439dd480a554 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:53:45.993393 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b7cdb2-0c1d-4900-8928-439dd480a554 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:53:45.994971 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b7cdb2-0c1d-4900-8928-439dd480a554 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:53:46.004286 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1165/2321] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 24 09:53:45 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:53:46.009111 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9662cce-2f09-435f-923d-966d52e1207f admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:53:46.019227 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9662cce-2f09-435f-923d-966d52e1207f admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:53:46.024428 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d236a0-6646-42fc-832b-cfc743701aa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:53:46.025079 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d236a0-6646-42fc-832b-cfc743701aa9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:53:46.025360 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d236a0-6646-42fc-832b-cfc743701aa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:53:46.025641 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d236a0-6646-42fc-832b-cfc743701aa9 admin admin] PATH_INFO: `/v3/projects/a192611a0d104261ae90599349375c08/users/8182ed6e1e904a5b903f5d3ae8211b4e/roles/15f634d4a2d247ddab90c458e5e1c141` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:53:46.028132 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d236a0-6646-42fc-832b-cfc743701aa9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a192611a0d104261ae90599349375c08, user_id=8182ed6e1e904a5b903f5d3ae8211b4e, role_id=15f634d4a2d247ddab90c458e5e1c141)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:53:46.030754 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d236a0-6646-42fc-832b-cfc743701aa9 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:53:46.041474 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1157/2322] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 24 09:53:46 2022] PUT /identity/v3/projects/a192611a0d104261ae90599349375c08/users/8182ed6e1e904a5b903f5d3ae8211b4e/roles/15f634d4a2d247ddab90c458e5e1c141 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:53:46.106373 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1ac6fa-bb24-4d26-acae-5bc0a67cd4ed None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:53:46.106640 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1ac6fa-bb24-4d26-acae-5bc0a67cd4ed None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:53:46.106918 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1ac6fa-bb24-4d26-acae-5bc0a67cd4ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:53:46.134111 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-ff1ac6fa-bb24-4d26-acae-5bc0a67cd4ed None None] MFA Rules not processed for user `8182ed6e1e904a5b903f5d3ae8211b4e`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:53:46.135338 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-ff1ac6fa-bb24-4d26-acae-5bc0a67cd4ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:53:46.199167 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:53:46.202621 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1166/2323] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 09:53:46 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:53:56.037786 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=30,keystone:INSERT=5 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:54:24.786596 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d236a0-6646-42fc-832b-cfc743701aa9 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:54:24.788618 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-f2d236a0-6646-42fc-832b-cfc743701aa9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:54:24.845670 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d236a0-6646-42fc-832b-cfc743701aa9 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:54:24.849262 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a40f188-c691-4012-b3ac-57adf766341f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e5260e798404e8bb12b25ca8ea97052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:54:24.849864 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a40f188-c691-4012-b3ac-57adf766341f service neutron] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:54:24.850139 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a40f188-c691-4012-b3ac-57adf766341f service neutron] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:54:24.850407 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a40f188-c691-4012-b3ac-57adf766341f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:54:24.852168 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-4a40f188-c691-4012-b3ac-57adf766341f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:54:24.861876 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a40f188-c691-4012-b3ac-57adf766341f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:54:24.863273 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a40f188-c691-4012-b3ac-57adf766341f service neutron] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:54:24.871032 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1158/2324] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:54:24 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:54:34.867624 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=26 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:55:32.560546 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1ac6fa-bb24-4d26-acae-5bc0a67cd4ed None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:55:32.561959 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-ff1ac6fa-bb24-4d26-acae-5bc0a67cd4ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:55:32.654672 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1ac6fa-bb24-4d26-acae-5bc0a67cd4ed None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:55:32.658276 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2e3a04-4241-462b-94d8-7851746fd242 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c32b83577c46488906fe409a790b24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:55:32.658883 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2e3a04-4241-462b-94d8-7851746fd242 service cinder] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:55:32.659163 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2e3a04-4241-462b-94d8-7851746fd242 service cinder] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:55:32.659438 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2e3a04-4241-462b-94d8-7851746fd242 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:55:32.661226 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-be2e3a04-4241-462b-94d8-7851746fd242 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:55:32.731067 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2e3a04-4241-462b-94d8-7851746fd242 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:55:32.732534 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2e3a04-4241-462b-94d8-7851746fd242 service cinder] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:55:32.741484 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1167/2325] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:55:32 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:55:32.784783 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a40f188-c691-4012-b3ac-57adf766341f service neutron] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:55:32.809251 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a40f188-c691-4012-b3ac-57adf766341f service neutron] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:55:32.809251 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79ff968-32c3-4d92-bf59-2f1ff74767c2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c32b83577c46488906fe409a790b24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:55:32.809251 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79ff968-32c3-4d92-bf59-2f1ff74767c2 service cinder] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:55:32.809251 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79ff968-32c3-4d92-bf59-2f1ff74767c2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:55:32.809748 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79ff968-32c3-4d92-bf59-2f1ff74767c2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:55:32.810684 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-c79ff968-32c3-4d92-bf59-2f1ff74767c2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:55:32.870135 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79ff968-32c3-4d92-bf59-2f1ff74767c2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:55:32.871564 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79ff968-32c3-4d92-bf59-2f1ff74767c2 service cinder] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:55:32.880830 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1159/2326] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:55:32 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:55:32.951088 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2e3a04-4241-462b-94d8-7851746fd242 service cinder] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:55:32.959344 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2e3a04-4241-462b-94d8-7851746fd242 service cinder] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:55:32.962946 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63af0bea-23d6-4585-86d5-10e12007d690 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c32b83577c46488906fe409a790b24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:55:32.963544 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63af0bea-23d6-4585-86d5-10e12007d690 service cinder] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:55:32.963859 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63af0bea-23d6-4585-86d5-10e12007d690 service cinder] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:55:32.964154 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63af0bea-23d6-4585-86d5-10e12007d690 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:55:32.974244 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63af0bea-23d6-4585-86d5-10e12007d690 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:55:32.975667 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63af0bea-23d6-4585-86d5-10e12007d690 service cinder] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:55:32.984966 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1168/2327] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:55:32 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:55:42.876510 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=33 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:55:42.980732 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=77 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:20.092437 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79ff968-32c3-4d92-bf59-2f1ff74767c2 service cinder] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:20.093895 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-c79ff968-32c3-4d92-bf59-2f1ff74767c2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:57:20.189358 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79ff968-32c3-4d92-bf59-2f1ff74767c2 service cinder] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:20.194357 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5a2548-381f-4dfb-a362-23cc6b90d469 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80cada22039d4f11935412ee578acd05', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:20.194960 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5a2548-381f-4dfb-a362-23cc6b90d469 service placement] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:20.195241 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5a2548-381f-4dfb-a362-23cc6b90d469 service placement] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:20.195515 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5a2548-381f-4dfb-a362-23cc6b90d469 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:20.197258 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-ba5a2548-381f-4dfb-a362-23cc6b90d469 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:57:20.209358 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5a2548-381f-4dfb-a362-23cc6b90d469 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:20.210958 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5a2548-381f-4dfb-a362-23cc6b90d469 service placement] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:20.220907 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1160/2328] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:57:20 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:57:30.216753 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=44 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:33.340018 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63af0bea-23d6-4585-86d5-10e12007d690 service cinder] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:33.349982 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63af0bea-23d6-4585-86d5-10e12007d690 service cinder] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:33.353587 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f1329e-cc87-452b-8627-3214e6f39d88 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c32b83577c46488906fe409a790b24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:33.354352 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f1329e-cc87-452b-8627-3214e6f39d88 service cinder] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:33.354619 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f1329e-cc87-452b-8627-3214e6f39d88 service cinder] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:33.354893 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f1329e-cc87-452b-8627-3214e6f39d88 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:33.356653 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-39f1329e-cc87-452b-8627-3214e6f39d88 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:57:33.414287 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f1329e-cc87-452b-8627-3214e6f39d88 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:33.415735 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f1329e-cc87-452b-8627-3214e6f39d88 service cinder] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:33.434448 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1169/2329] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:57:33 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:57:33.730996 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5a2548-381f-4dfb-a362-23cc6b90d469 service placement] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:33.798497 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5a2548-381f-4dfb-a362-23cc6b90d469 service placement] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:33.803657 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb58ef7-052d-4bf1-9c73-4f9058037cde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:33.804299 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb58ef7-052d-4bf1-9c73-4f9058037cde admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:33.819196 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb58ef7-052d-4bf1-9c73-4f9058037cde admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:33.819196 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb58ef7-052d-4bf1-9c73-4f9058037cde admin admin] PATH_INFO: `/v3/users/48b02582c40e49ee9f1f3b8ced324d16` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:33.819196 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb58ef7-052d-4bf1-9c73-4f9058037cde admin admin] RBAC: Authorizing `identity:delete_user(user_id=48b02582c40e49ee9f1f3b8ced324d16)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:33.819196 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb58ef7-052d-4bf1-9c73-4f9058037cde admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:33.945755 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-4bb58ef7-052d-4bf1-9c73-4f9058037cde admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '48b02582c40e49ee9f1f3b8ced324d16'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:57:33.946411 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-4bb58ef7-052d-4bf1-9c73-4f9058037cde admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '48b02582c40e49ee9f1f3b8ced324d16'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:57:33.960071 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-4bb58ef7-052d-4bf1-9c73-4f9058037cde admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '48b02582c40e49ee9f1f3b8ced324d16'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:57:33.967189 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-4bb58ef7-052d-4bf1-9c73-4f9058037cde admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '48b02582c40e49ee9f1f3b8ced324d16'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:57:34.012355 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-4bb58ef7-052d-4bf1-9c73-4f9058037cde admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '48b02582c40e49ee9f1f3b8ced324d16'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:57:34.018539 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1161/2330] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 24 09:57:33 2022] DELETE /identity/v3/users/48b02582c40e49ee9f1f3b8ced324d16 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:57:34.024076 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c205fcc6-121f-44fd-8db3-cc56b2f9ba93 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:34.024444 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c205fcc6-121f-44fd-8db3-cc56b2f9ba93 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:34.024796 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c205fcc6-121f-44fd-8db3-cc56b2f9ba93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:34.046267 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-c205fcc6-121f-44fd-8db3-cc56b2f9ba93 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:57:34.047514 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-c205fcc6-121f-44fd-8db3-cc56b2f9ba93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:57:34.107538 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1170/2331] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 09:57:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:57:34.117649 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb58ef7-052d-4bf1-9c73-4f9058037cde admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:34.119140 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-4bb58ef7-052d-4bf1-9c73-4f9058037cde admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:57:34.180134 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb58ef7-052d-4bf1-9c73-4f9058037cde admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:34.184035 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a120c8b8-9f69-4e89-9ec0-c06ff08a6675 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e5260e798404e8bb12b25ca8ea97052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:34.184645 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a120c8b8-9f69-4e89-9ec0-c06ff08a6675 service neutron] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:34.184924 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a120c8b8-9f69-4e89-9ec0-c06ff08a6675 service neutron] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:34.185198 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a120c8b8-9f69-4e89-9ec0-c06ff08a6675 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:34.194616 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a120c8b8-9f69-4e89-9ec0-c06ff08a6675 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:34.196072 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a120c8b8-9f69-4e89-9ec0-c06ff08a6675 service neutron] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:34.205390 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1162/2332] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:57:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:57:34.811294 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c205fcc6-121f-44fd-8db3-cc56b2f9ba93 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:34.811294 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-c205fcc6-121f-44fd-8db3-cc56b2f9ba93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:57:34.811294 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c205fcc6-121f-44fd-8db3-cc56b2f9ba93 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:34.811985 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f7d6ef-3559-4db8-8942-1dfcf015fd73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:34.811985 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f7d6ef-3559-4db8-8942-1dfcf015fd73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:34.811985 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f7d6ef-3559-4db8-8942-1dfcf015fd73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:34.811985 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f7d6ef-3559-4db8-8942-1dfcf015fd73 admin admin] PATH_INFO: `/v3/projects/1ba34edd65e546edb5d360f0bcec102c` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:34.813130 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f7d6ef-3559-4db8-8942-1dfcf015fd73 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1ba34edd65e546edb5d360f0bcec102c)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:34.815043 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f7d6ef-3559-4db8-8942-1dfcf015fd73 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:34.945188 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-18f7d6ef-3559-4db8-8942-1dfcf015fd73 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1ba34edd65e546edb5d360f0bcec102c', 'request_id': 'req-18f7d6ef-3559-4db8-8942-1dfcf015fd73'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:57:34.951544 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-18f7d6ef-3559-4db8-8942-1dfcf015fd73 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1ba34edd65e546edb5d360f0bcec102c', 'request_id': 'req-18f7d6ef-3559-4db8-8942-1dfcf015fd73'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:57:34.958252 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-18f7d6ef-3559-4db8-8942-1dfcf015fd73 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1ba34edd65e546edb5d360f0bcec102c', 'request_id': 'req-18f7d6ef-3559-4db8-8942-1dfcf015fd73'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:57:35.064940 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-18f7d6ef-3559-4db8-8942-1dfcf015fd73 admin admin] The token cache is being invalidate because project 1ba34edd65e546edb5d360f0bcec102c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81602) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:57:35.065234 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-18f7d6ef-3559-4db8-8942-1dfcf015fd73 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:57:35.065861 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-18f7d6ef-3559-4db8-8942-1dfcf015fd73 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:57:35.070998 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1171/2333] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 24 09:57:34 2022] DELETE /identity/v3/projects/1ba34edd65e546edb5d360f0bcec102c => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:57:35.072153 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a120c8b8-9f69-4e89-9ec0-c06ff08a6675 service neutron] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:35.083548 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a120c8b8-9f69-4e89-9ec0-c06ff08a6675 service neutron] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:35.104207 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:35.150335 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1cd6a0-e560-4948-871c-59a7dae1468f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:35.151087 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1cd6a0-e560-4948-871c-59a7dae1468f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:35.151369 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1cd6a0-e560-4948-871c-59a7dae1468f admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:35.151678 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1cd6a0-e560-4948-871c-59a7dae1468f admin admin] PATH_INFO: `/v3/users/8182ed6e1e904a5b903f5d3ae8211b4e` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:35.152816 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1cd6a0-e560-4948-871c-59a7dae1468f admin admin] RBAC: Authorizing `identity:delete_user(user_id=8182ed6e1e904a5b903f5d3ae8211b4e)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:35.154814 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1cd6a0-e560-4948-871c-59a7dae1468f admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:35.346297 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-ff1cd6a0-e560-4948-871c-59a7dae1468f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8182ed6e1e904a5b903f5d3ae8211b4e'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:57:35.347029 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-ff1cd6a0-e560-4948-871c-59a7dae1468f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8182ed6e1e904a5b903f5d3ae8211b4e'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:57:35.358604 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-ff1cd6a0-e560-4948-871c-59a7dae1468f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8182ed6e1e904a5b903f5d3ae8211b4e'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:57:35.365848 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-ff1cd6a0-e560-4948-871c-59a7dae1468f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8182ed6e1e904a5b903f5d3ae8211b4e'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:57:35.423453 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-ff1cd6a0-e560-4948-871c-59a7dae1468f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8182ed6e1e904a5b903f5d3ae8211b4e'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:57:35.430174 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1163/2334] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 24 09:57:35 2022] DELETE /identity/v3/users/8182ed6e1e904a5b903f5d3ae8211b4e => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 09:57:35.796690 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f7d6ef-3559-4db8-8942-1dfcf015fd73 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:35.798328 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-18f7d6ef-3559-4db8-8942-1dfcf015fd73 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:57:35.813784 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f7d6ef-3559-4db8-8942-1dfcf015fd73 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:35.826149 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:35.873005 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b81e2f-2e07-45d2-9b47-9d0bf643c5aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:35.873563 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b81e2f-2e07-45d2-9b47-9d0bf643c5aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:35.873847 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b81e2f-2e07-45d2-9b47-9d0bf643c5aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:35.874123 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b81e2f-2e07-45d2-9b47-9d0bf643c5aa admin admin] PATH_INFO: `/v3/projects/a192611a0d104261ae90599349375c08` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:35.875237 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b81e2f-2e07-45d2-9b47-9d0bf643c5aa admin admin] RBAC: Authorizing `identity:delete_project(project_id=a192611a0d104261ae90599349375c08)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:35.877190 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b81e2f-2e07-45d2-9b47-9d0bf643c5aa admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:35.978170 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-82b81e2f-2e07-45d2-9b47-9d0bf643c5aa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a192611a0d104261ae90599349375c08', 'request_id': 'req-82b81e2f-2e07-45d2-9b47-9d0bf643c5aa'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:57:35.984322 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-82b81e2f-2e07-45d2-9b47-9d0bf643c5aa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a192611a0d104261ae90599349375c08', 'request_id': 'req-82b81e2f-2e07-45d2-9b47-9d0bf643c5aa'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:57:35.990403 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-82b81e2f-2e07-45d2-9b47-9d0bf643c5aa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a192611a0d104261ae90599349375c08', 'request_id': 'req-82b81e2f-2e07-45d2-9b47-9d0bf643c5aa'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:57:36.080327 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-82b81e2f-2e07-45d2-9b47-9d0bf643c5aa admin admin] The token cache is being invalidate because project a192611a0d104261ae90599349375c08 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81602) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:57:36.080622 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-82b81e2f-2e07-45d2-9b47-9d0bf643c5aa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:57:36.081502 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-82b81e2f-2e07-45d2-9b47-9d0bf643c5aa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:57:36.083087 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1172/2335] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 24 09:57:35 2022] DELETE /identity/v3/projects/a192611a0d104261ae90599349375c08 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:57:36.087902 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634f9c4a-c728-4b8e-9ede-8f288313c785 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:36.090592 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634f9c4a-c728-4b8e-9ede-8f288313c785 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:36.090592 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634f9c4a-c728-4b8e-9ede-8f288313c785 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:36.107637 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-634f9c4a-c728-4b8e-9ede-8f288313c785 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:57:36.108836 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-634f9c4a-c728-4b8e-9ede-8f288313c785 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:57:36.167566 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1164/2336] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 09:57:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:57:36.175138 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfde1573-4755-402d-a29a-f51742b8431d None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:36.175994 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfde1573-4755-402d-a29a-f51742b8431d None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:36.176270 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfde1573-4755-402d-a29a-f51742b8431d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:36.197843 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-dfde1573-4755-402d-a29a-f51742b8431d None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:57:36.199063 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-dfde1573-4755-402d-a29a-f51742b8431d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:57:36.207213 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1173/2337] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 09:57:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 09:57:36.211411 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634f9c4a-c728-4b8e-9ede-8f288313c785 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:36.221178 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634f9c4a-c728-4b8e-9ede-8f288313c785 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:36.225734 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b92ce43-43b6-4558-8c89-3d8acc0b034b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:36.226334 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b92ce43-43b6-4558-8c89-3d8acc0b034b admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:36.226616 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b92ce43-43b6-4558-8c89-3d8acc0b034b admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:36.226889 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b92ce43-43b6-4558-8c89-3d8acc0b034b admin admin] PATH_INFO: `/v3/domains` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:36.227535 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b92ce43-43b6-4558-8c89-3d8acc0b034b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:57:36.227872 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b92ce43-43b6-4558-8c89-3d8acc0b034b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:36.229467 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b92ce43-43b6-4558-8c89-3d8acc0b034b admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:36.241194 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1165/2338] 10.222.0.16 () {58 vars in 1153 bytes} [Wed Aug 24 09:57:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:57:36.245298 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfde1573-4755-402d-a29a-f51742b8431d None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:36.254997 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfde1573-4755-402d-a29a-f51742b8431d None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:36.259615 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06ac8b7-8729-445c-9de4-d65a951e674d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:36.260198 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06ac8b7-8729-445c-9de4-d65a951e674d admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:36.260474 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06ac8b7-8729-445c-9de4-d65a951e674d admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:36.260744 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06ac8b7-8729-445c-9de4-d65a951e674d admin admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:36.261355 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06ac8b7-8729-445c-9de4-d65a951e674d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:36.263294 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06ac8b7-8729-445c-9de4-d65a951e674d admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:36.368382 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1174/2339] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 24 09:57:36 2022] POST /identity/v3/projects => generated 381 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:57:36.372533 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b92ce43-43b6-4558-8c89-3d8acc0b034b admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:36.383139 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b92ce43-43b6-4558-8c89-3d8acc0b034b admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:36.408854 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:36.437418 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aeddbfd-a6bc-4ed6-af26-8e37b45f53d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:36.438051 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aeddbfd-a6bc-4ed6-af26-8e37b45f53d6 admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:36.438333 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aeddbfd-a6bc-4ed6-af26-8e37b45f53d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:36.438608 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aeddbfd-a6bc-4ed6-af26-8e37b45f53d6 admin admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:36.439226 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aeddbfd-a6bc-4ed6-af26-8e37b45f53d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:36.441160 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aeddbfd-a6bc-4ed6-af26-8e37b45f53d6 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:36.526646 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1166/2340] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 24 09:57:36 2022] POST /identity/v3/users => generated 291 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:57:36.530930 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06ac8b7-8729-445c-9de4-d65a951e674d admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:36.541280 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06ac8b7-8729-445c-9de4-d65a951e674d admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:36.545951 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00fb96e-babc-4bc4-80e7-7f3a270eb2b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:36.546548 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00fb96e-babc-4bc4-80e7-7f3a270eb2b5 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:36.546940 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00fb96e-babc-4bc4-80e7-7f3a270eb2b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:36.547216 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00fb96e-babc-4bc4-80e7-7f3a270eb2b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:36.547889 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00fb96e-babc-4bc4-80e7-7f3a270eb2b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:36.549482 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00fb96e-babc-4bc4-80e7-7f3a270eb2b5 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:36.559695 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1175/2341] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 24 09:57:36 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:57:36.563521 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aeddbfd-a6bc-4ed6-af26-8e37b45f53d6 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:36.573791 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aeddbfd-a6bc-4ed6-af26-8e37b45f53d6 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:36.578343 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4d0185-8f98-4e90-897a-c3c9c20151cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:36.578931 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4d0185-8f98-4e90-897a-c3c9c20151cd admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:36.579213 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4d0185-8f98-4e90-897a-c3c9c20151cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:36.579487 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4d0185-8f98-4e90-897a-c3c9c20151cd admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:36.580155 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4d0185-8f98-4e90-897a-c3c9c20151cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:36.581996 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4d0185-8f98-4e90-897a-c3c9c20151cd admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:36.592865 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1167/2342] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 24 09:57:36 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:57:36.596740 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00fb96e-babc-4bc4-80e7-7f3a270eb2b5 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:36.606552 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00fb96e-babc-4bc4-80e7-7f3a270eb2b5 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:36.611169 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1baff45e-316c-4b97-b6ae-f97c77986828 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:36.611830 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baff45e-316c-4b97-b6ae-f97c77986828 admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:36.612106 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baff45e-316c-4b97-b6ae-f97c77986828 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:36.612374 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baff45e-316c-4b97-b6ae-f97c77986828 admin admin] PATH_INFO: `/v3/projects/885df5002d584d3e851b890198f21550/users/93a6cbe2a558474aa943682967166deb/roles/136cdf78e00a426997dd4e37faff489b` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:36.626232 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1baff45e-316c-4b97-b6ae-f97c77986828 admin admin] RBAC: Authorizing `identity:create_grant(project_id=885df5002d584d3e851b890198f21550, user_id=93a6cbe2a558474aa943682967166deb, role_id=136cdf78e00a426997dd4e37faff489b)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:36.628875 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1baff45e-316c-4b97-b6ae-f97c77986828 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:36.693009 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1176/2343] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 24 09:57:36 2022] PUT /identity/v3/projects/885df5002d584d3e851b890198f21550/users/93a6cbe2a558474aa943682967166deb/roles/136cdf78e00a426997dd4e37faff489b => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:57:36.703043 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c8fda9-5d59-4c4b-b058-d6e9a38387b7 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:36.703555 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c8fda9-5d59-4c4b-b058-d6e9a38387b7 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:36.703860 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c8fda9-5d59-4c4b-b058-d6e9a38387b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:36.732324 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-45c8fda9-5d59-4c4b-b058-d6e9a38387b7 None None] MFA Rules not processed for user `93a6cbe2a558474aa943682967166deb`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:57:36.733711 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-45c8fda9-5d59-4c4b-b058-d6e9a38387b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:57:36.803534 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1168/2344] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 09:57:36 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:57:36.808036 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1baff45e-316c-4b97-b6ae-f97c77986828 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:36.849840 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:36.869347 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1baff45e-316c-4b97-b6ae-f97c77986828 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:36.874088 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d570d9f3-66c0-44a7-a67a-60be67868047 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:36.874902 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d570d9f3-66c0-44a7-a67a-60be67868047 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:36.875400 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d570d9f3-66c0-44a7-a67a-60be67868047 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:36.875736 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d570d9f3-66c0-44a7-a67a-60be67868047 admin admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:36.876844 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d570d9f3-66c0-44a7-a67a-60be67868047 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:36.878825 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d570d9f3-66c0-44a7-a67a-60be67868047 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:36.939359 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1177/2345] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 24 09:57:36 2022] POST /identity/v3/projects => generated 379 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:57:36.944030 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c8fda9-5d59-4c4b-b058-d6e9a38387b7 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:36.954863 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c8fda9-5d59-4c4b-b058-d6e9a38387b7 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:37.015979 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e9afee-4f9a-424b-bf0f-94d4d233b7bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:37.016588 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e9afee-4f9a-424b-bf0f-94d4d233b7bd admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:37.016871 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e9afee-4f9a-424b-bf0f-94d4d233b7bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:37.017145 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e9afee-4f9a-424b-bf0f-94d4d233b7bd admin admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:37.017781 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e9afee-4f9a-424b-bf0f-94d4d233b7bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:37.019778 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e9afee-4f9a-424b-bf0f-94d4d233b7bd admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:37.116539 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1169/2346] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 24 09:57:36 2022] POST /identity/v3/users => generated 296 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:57:37.120568 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d570d9f3-66c0-44a7-a67a-60be67868047 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:37.130776 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d570d9f3-66c0-44a7-a67a-60be67868047 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:37.135407 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f574d71f-cf20-4757-a070-32fda48da9b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:37.136105 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f574d71f-cf20-4757-a070-32fda48da9b8 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:37.136383 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f574d71f-cf20-4757-a070-32fda48da9b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:37.136654 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f574d71f-cf20-4757-a070-32fda48da9b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:37.137285 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f574d71f-cf20-4757-a070-32fda48da9b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:37.138853 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f574d71f-cf20-4757-a070-32fda48da9b8 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:37.148706 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1178/2347] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 24 09:57:37 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:57:37.152562 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e9afee-4f9a-424b-bf0f-94d4d233b7bd admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:37.162922 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e9afee-4f9a-424b-bf0f-94d4d233b7bd admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:37.167558 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e397b53-a551-4a48-b3b5-8054acfefccc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:37.168236 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e397b53-a551-4a48-b3b5-8054acfefccc admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:37.168518 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e397b53-a551-4a48-b3b5-8054acfefccc admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:37.168791 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e397b53-a551-4a48-b3b5-8054acfefccc admin admin] PATH_INFO: `/v3/projects/f6ea09160bff441aa713ab604e219575/users/ca9d815de9e54796ae265d53e528b239/roles/15f634d4a2d247ddab90c458e5e1c141` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:37.184873 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e397b53-a551-4a48-b3b5-8054acfefccc admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6ea09160bff441aa713ab604e219575, user_id=ca9d815de9e54796ae265d53e528b239, role_id=15f634d4a2d247ddab90c458e5e1c141)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:37.187512 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e397b53-a551-4a48-b3b5-8054acfefccc admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:37.259551 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1170/2348] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 24 09:57:37 2022] PUT /identity/v3/projects/f6ea09160bff441aa713ab604e219575/users/ca9d815de9e54796ae265d53e528b239/roles/15f634d4a2d247ddab90c458e5e1c141 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:57:37.265809 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f574d71f-cf20-4757-a070-32fda48da9b8 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:37.277238 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f574d71f-cf20-4757-a070-32fda48da9b8 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:37.334387 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a4b87c-6daa-4fa7-bb59-096cce8248a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:37.335000 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a4b87c-6daa-4fa7-bb59-096cce8248a9 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:37.335282 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a4b87c-6daa-4fa7-bb59-096cce8248a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:37.335557 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a4b87c-6daa-4fa7-bb59-096cce8248a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:37.336267 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a4b87c-6daa-4fa7-bb59-096cce8248a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:37.337858 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a4b87c-6daa-4fa7-bb59-096cce8248a9 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:37.347536 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1179/2349] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 24 09:57:37 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:57:37.351806 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e397b53-a551-4a48-b3b5-8054acfefccc admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:37.357191 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:37.362314 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e397b53-a551-4a48-b3b5-8054acfefccc admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:37.366906 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a51d94-173f-437c-9f3b-d4ce9f8aa583 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:37.367929 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a51d94-173f-437c-9f3b-d4ce9f8aa583 admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:37.367929 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a51d94-173f-437c-9f3b-d4ce9f8aa583 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:37.368161 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a51d94-173f-437c-9f3b-d4ce9f8aa583 admin admin] PATH_INFO: `/v3/projects/f6ea09160bff441aa713ab604e219575/users/ca9d815de9e54796ae265d53e528b239/roles/15f634d4a2d247ddab90c458e5e1c141` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:37.370029 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a51d94-173f-437c-9f3b-d4ce9f8aa583 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6ea09160bff441aa713ab604e219575, user_id=ca9d815de9e54796ae265d53e528b239, role_id=15f634d4a2d247ddab90c458e5e1c141)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:37.372722 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a51d94-173f-437c-9f3b-d4ce9f8aa583 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:37.384220 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1171/2350] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 24 09:57:37 2022] PUT /identity/v3/projects/f6ea09160bff441aa713ab604e219575/users/ca9d815de9e54796ae265d53e528b239/roles/15f634d4a2d247ddab90c458e5e1c141 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:57:37.394230 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b2d5aa-d795-48be-9a37-e14165328048 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:37.394517 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b2d5aa-d795-48be-9a37-e14165328048 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:37.395368 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b2d5aa-d795-48be-9a37-e14165328048 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:37.422137 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-e9b2d5aa-d795-48be-9a37-e14165328048 None None] MFA Rules not processed for user `ca9d815de9e54796ae265d53e528b239`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:57:37.423340 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-e9b2d5aa-d795-48be-9a37-e14165328048 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:57:37.489658 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1180/2351] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 09:57:37 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:57:37.500611 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a51d94-173f-437c-9f3b-d4ce9f8aa583 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:37.502304 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-a2a51d94-173f-437c-9f3b-d4ce9f8aa583 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:57:37.558050 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a51d94-173f-437c-9f3b-d4ce9f8aa583 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:37.561677 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127e0c6d-ba97-4f90-ad26-86509fffc619 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c32b83577c46488906fe409a790b24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:37.562264 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127e0c6d-ba97-4f90-ad26-86509fffc619 service cinder] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:37.562538 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127e0c6d-ba97-4f90-ad26-86509fffc619 service cinder] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:37.562804 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127e0c6d-ba97-4f90-ad26-86509fffc619 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:37.572100 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127e0c6d-ba97-4f90-ad26-86509fffc619 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:37.573511 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127e0c6d-ba97-4f90-ad26-86509fffc619 service cinder] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:37.583756 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1172/2352] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:57:37 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:57:37.799159 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b2d5aa-d795-48be-9a37-e14165328048 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:37.809858 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b2d5aa-d795-48be-9a37-e14165328048 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:37.813432 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7508a16f-6183-4660-940c-34d286041f73 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c32b83577c46488906fe409a790b24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:37.816369 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7508a16f-6183-4660-940c-34d286041f73 service cinder] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:37.816369 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7508a16f-6183-4660-940c-34d286041f73 service cinder] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:37.816369 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7508a16f-6183-4660-940c-34d286041f73 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:37.847030 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:37.872152 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7508a16f-6183-4660-940c-34d286041f73 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:37.873596 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7508a16f-6183-4660-940c-34d286041f73 service cinder] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:37.882799 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1181/2353] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:57:37 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:57:47.578840 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=30,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:47.960560 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=12 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:01.204135 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127e0c6d-ba97-4f90-ad26-86509fffc619 service cinder] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:01.213615 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127e0c6d-ba97-4f90-ad26-86509fffc619 service cinder] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:01.217270 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a6fcdd-688e-4a5f-be2d-57637e95052c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c32b83577c46488906fe409a790b24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:01.217865 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a6fcdd-688e-4a5f-be2d-57637e95052c service cinder] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:01.218147 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a6fcdd-688e-4a5f-be2d-57637e95052c service cinder] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:01.218420 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a6fcdd-688e-4a5f-be2d-57637e95052c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:01.220181 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-17a6fcdd-688e-4a5f-be2d-57637e95052c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:58:01.278167 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a6fcdd-688e-4a5f-be2d-57637e95052c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:01.279621 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a6fcdd-688e-4a5f-be2d-57637e95052c service cinder] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:01.288570 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1173/2354] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:58:01 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:58:11.284287 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=28 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:59:28.769500 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7508a16f-6183-4660-940c-34d286041f73 service cinder] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:28.771021 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-7508a16f-6183-4660-940c-34d286041f73 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:59:28.838020 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7508a16f-6183-4660-940c-34d286041f73 service cinder] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:28.841498 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56e2b1f-99dc-4959-a788-dffa164927ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e5260e798404e8bb12b25ca8ea97052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:28.842096 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56e2b1f-99dc-4959-a788-dffa164927ae service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:28.842368 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56e2b1f-99dc-4959-a788-dffa164927ae service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:28.842635 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56e2b1f-99dc-4959-a788-dffa164927ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:28.844404 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-e56e2b1f-99dc-4959-a788-dffa164927ae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:59:28.853825 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56e2b1f-99dc-4959-a788-dffa164927ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:59:28.855277 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56e2b1f-99dc-4959-a788-dffa164927ae service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:59:28.863079 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1182/2355] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:59:28 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:59:33.043639 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a6fcdd-688e-4a5f-be2d-57637e95052c service cinder] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:33.043639 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-17a6fcdd-688e-4a5f-be2d-57637e95052c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:59:33.070152 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a6fcdd-688e-4a5f-be2d-57637e95052c service cinder] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:33.075383 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e9b49b-235e-4752-9efe-828be9e2c2d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e5260e798404e8bb12b25ca8ea97052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:33.075383 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e9b49b-235e-4752-9efe-828be9e2c2d5 service neutron] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:33.075383 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e9b49b-235e-4752-9efe-828be9e2c2d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:33.076891 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e9b49b-235e-4752-9efe-828be9e2c2d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:33.078857 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-d2e9b49b-235e-4752-9efe-828be9e2c2d5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:59:33.090323 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56e2b1f-99dc-4959-a788-dffa164927ae service neutron] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:33.099054 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56e2b1f-99dc-4959-a788-dffa164927ae service neutron] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:33.103548 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e4659f-dc2e-4414-823d-8934b6fd675a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e5260e798404e8bb12b25ca8ea97052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:33.109375 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e4659f-dc2e-4414-823d-8934b6fd675a service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:33.109375 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e4659f-dc2e-4414-823d-8934b6fd675a service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:33.109375 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e4659f-dc2e-4414-823d-8934b6fd675a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:33.109375 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-d1e4659f-dc2e-4414-823d-8934b6fd675a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:59:33.169205 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e9b49b-235e-4752-9efe-828be9e2c2d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:59:33.170643 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e9b49b-235e-4752-9efe-828be9e2c2d5 service neutron] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:59:33.172449 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e4659f-dc2e-4414-823d-8934b6fd675a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:59:33.173901 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e4659f-dc2e-4414-823d-8934b6fd675a service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:59:33.189796 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1174/2356] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:59:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:59:33.193095 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1183/2357] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:59:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 09:59:37.946986 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e9b49b-235e-4752-9efe-828be9e2c2d5 service neutron] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:37.948707 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-d2e9b49b-235e-4752-9efe-828be9e2c2d5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:59:38.059885 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e9b49b-235e-4752-9efe-828be9e2c2d5 service neutron] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:38.065754 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832414dc-ed3e-4f95-95fe-6beb19970c77 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80cada22039d4f11935412ee578acd05', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:38.066512 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832414dc-ed3e-4f95-95fe-6beb19970c77 service placement] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:38.066931 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832414dc-ed3e-4f95-95fe-6beb19970c77 service placement] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:38.067311 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832414dc-ed3e-4f95-95fe-6beb19970c77 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:38.069354 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-832414dc-ed3e-4f95-95fe-6beb19970c77 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:59:38.082597 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832414dc-ed3e-4f95-95fe-6beb19970c77 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:59:38.084316 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832414dc-ed3e-4f95-95fe-6beb19970c77 service placement] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:59:38.113166 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1175/2358] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:59:37 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:59:38.118847 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e4659f-dc2e-4414-823d-8934b6fd675a service neutron] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:38.120361 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-d1e4659f-dc2e-4414-823d-8934b6fd675a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:59:38.141088 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e4659f-dc2e-4414-823d-8934b6fd675a service neutron] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:38.148195 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d36343-cc50-4558-81d5-aa6ddf2b63ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:38.150163 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d36343-cc50-4558-81d5-aa6ddf2b63ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:38.151388 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d36343-cc50-4558-81d5-aa6ddf2b63ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:38.151984 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d36343-cc50-4558-81d5-aa6ddf2b63ac admin admin] PATH_INFO: `/v3/users/b9099dc3ad92431cbac98f144690038c` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:38.176304 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d36343-cc50-4558-81d5-aa6ddf2b63ac admin admin] RBAC: Authorizing `identity:delete_user(user_id=b9099dc3ad92431cbac98f144690038c)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:59:38.178297 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d36343-cc50-4558-81d5-aa6ddf2b63ac admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:59:38.275305 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832414dc-ed3e-4f95-95fe-6beb19970c77 service placement] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:38.306708 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832414dc-ed3e-4f95-95fe-6beb19970c77 service placement] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:38.318007 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9438659-88a9-4270-b26a-978c4a9a8e12 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80cada22039d4f11935412ee578acd05', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:38.318974 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9438659-88a9-4270-b26a-978c4a9a8e12 service placement] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:38.319746 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9438659-88a9-4270-b26a-978c4a9a8e12 service placement] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:38.321018 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9438659-88a9-4270-b26a-978c4a9a8e12 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:38.324617 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-d9438659-88a9-4270-b26a-978c4a9a8e12 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:59:38.350078 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9438659-88a9-4270-b26a-978c4a9a8e12 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:59:38.352834 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9438659-88a9-4270-b26a-978c4a9a8e12 service placement] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:59:38.374677 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1176/2359] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:59:38 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:59:38.391639 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-51d36343-cc50-4558-81d5-aa6ddf2b63ac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b9099dc3ad92431cbac98f144690038c'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:59:38.392391 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-51d36343-cc50-4558-81d5-aa6ddf2b63ac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b9099dc3ad92431cbac98f144690038c'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:59:38.482248 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9438659-88a9-4270-b26a-978c4a9a8e12 service placement] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:38.484107 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-d9438659-88a9-4270-b26a-978c4a9a8e12 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:59:38.484218 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-51d36343-cc50-4558-81d5-aa6ddf2b63ac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b9099dc3ad92431cbac98f144690038c'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:59:38.484779 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-51d36343-cc50-4558-81d5-aa6ddf2b63ac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b9099dc3ad92431cbac98f144690038c'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:59:38.494819 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-51d36343-cc50-4558-81d5-aa6ddf2b63ac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b9099dc3ad92431cbac98f144690038c'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:59:38.496224 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:59:38.501218 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9438659-88a9-4270-b26a-978c4a9a8e12 service placement] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:38.506078 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e07d6f-5c07-4bb6-aa2d-3126c1fd8e4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:38.506764 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e07d6f-5c07-4bb6-aa2d-3126c1fd8e4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:38.507046 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e07d6f-5c07-4bb6-aa2d-3126c1fd8e4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:38.507496 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e07d6f-5c07-4bb6-aa2d-3126c1fd8e4d admin admin] PATH_INFO: `/v3/users/cbcaa84bc9c54d7aa4178d0174a5936e` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:38.519629 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1184/2360] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 24 09:59:38 2022] DELETE /identity/v3/users/b9099dc3ad92431cbac98f144690038c => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:59:38.525145 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e07d6f-5c07-4bb6-aa2d-3126c1fd8e4d admin admin] RBAC: Authorizing `identity:delete_user(user_id=cbcaa84bc9c54d7aa4178d0174a5936e)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:59:38.527303 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e07d6f-5c07-4bb6-aa2d-3126c1fd8e4d admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:59:38.545594 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:59:38.687013 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-22e07d6f-5c07-4bb6-aa2d-3126c1fd8e4d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cbcaa84bc9c54d7aa4178d0174a5936e'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:59:38.687805 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-22e07d6f-5c07-4bb6-aa2d-3126c1fd8e4d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cbcaa84bc9c54d7aa4178d0174a5936e'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:59:38.700065 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-22e07d6f-5c07-4bb6-aa2d-3126c1fd8e4d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cbcaa84bc9c54d7aa4178d0174a5936e'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:59:38.707114 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-22e07d6f-5c07-4bb6-aa2d-3126c1fd8e4d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cbcaa84bc9c54d7aa4178d0174a5936e'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:59:38.752451 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-22e07d6f-5c07-4bb6-aa2d-3126c1fd8e4d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cbcaa84bc9c54d7aa4178d0174a5936e'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:59:38.759820 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d36343-cc50-4558-81d5-aa6ddf2b63ac admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:38.761258 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-51d36343-cc50-4558-81d5-aa6ddf2b63ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:59:38.761820 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1177/2361] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 24 09:59:38 2022] DELETE /identity/v3/users/cbcaa84bc9c54d7aa4178d0174a5936e => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 09:59:38.834302 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d36343-cc50-4558-81d5-aa6ddf2b63ac admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:38.890972 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c0636f-6e62-4569-815d-d0b975ca2093 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:38.891638 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c0636f-6e62-4569-815d-d0b975ca2093 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:38.897343 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c0636f-6e62-4569-815d-d0b975ca2093 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:38.897343 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c0636f-6e62-4569-815d-d0b975ca2093 admin admin] PATH_INFO: `/v3/projects/803ce13612e0460682ef8b4d42781a99` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:38.903246 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c0636f-6e62-4569-815d-d0b975ca2093 admin admin] RBAC: Authorizing `identity:delete_project(project_id=803ce13612e0460682ef8b4d42781a99)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:59:38.905196 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c0636f-6e62-4569-815d-d0b975ca2093 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:59:39.030067 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e07d6f-5c07-4bb6-aa2d-3126c1fd8e4d admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:39.030067 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-22e07d6f-5c07-4bb6-aa2d-3126c1fd8e4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:59:39.043970 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e07d6f-5c07-4bb6-aa2d-3126c1fd8e4d admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:39.049482 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0017afae-3999-4381-8db7-d47723af9b4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:39.050141 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0017afae-3999-4381-8db7-d47723af9b4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:39.050731 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0017afae-3999-4381-8db7-d47723af9b4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:39.051020 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0017afae-3999-4381-8db7-d47723af9b4e admin admin] PATH_INFO: `/v3/projects/d00ce4a2669e43dcb5bc8a2d77b11c2f` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:39.064116 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0017afae-3999-4381-8db7-d47723af9b4e admin admin] RBAC: Authorizing `identity:delete_project(project_id=d00ce4a2669e43dcb5bc8a2d77b11c2f)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:59:39.066056 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0017afae-3999-4381-8db7-d47723af9b4e admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:59:39.072160 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-d6c0636f-6e62-4569-815d-d0b975ca2093 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '803ce13612e0460682ef8b4d42781a99', 'request_id': 'req-d6c0636f-6e62-4569-815d-d0b975ca2093'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:59:39.079086 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-d6c0636f-6e62-4569-815d-d0b975ca2093 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '803ce13612e0460682ef8b4d42781a99', 'request_id': 'req-d6c0636f-6e62-4569-815d-d0b975ca2093'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:59:39.085218 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-d6c0636f-6e62-4569-815d-d0b975ca2093 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '803ce13612e0460682ef8b4d42781a99', 'request_id': 'req-d6c0636f-6e62-4569-815d-d0b975ca2093'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:59:39.169936 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-d6c0636f-6e62-4569-815d-d0b975ca2093 admin admin] The token cache is being invalidate because project 803ce13612e0460682ef8b4d42781a99 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81602) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:59:39.170560 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-d6c0636f-6e62-4569-815d-d0b975ca2093 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:59:39.170913 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-d6c0636f-6e62-4569-815d-d0b975ca2093 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:59:39.172524 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1185/2362] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 24 09:59:38 2022] DELETE /identity/v3/projects/803ce13612e0460682ef8b4d42781a99 => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 09:59:39.178068 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b29b684-2d76-4e89-9310-163466e1c4a4 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:39.178356 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b29b684-2d76-4e89-9310-163466e1c4a4 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:39.178635 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b29b684-2d76-4e89-9310-163466e1c4a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:39.198494 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-0b29b684-2d76-4e89-9310-163466e1c4a4 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:59:39.199723 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-0b29b684-2d76-4e89-9310-163466e1c4a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:59:39.203972 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:59:39.206349 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-0017afae-3999-4381-8db7-d47723af9b4e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd00ce4a2669e43dcb5bc8a2d77b11c2f', 'request_id': 'req-0017afae-3999-4381-8db7-d47723af9b4e'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:59:39.212654 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-0017afae-3999-4381-8db7-d47723af9b4e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd00ce4a2669e43dcb5bc8a2d77b11c2f', 'request_id': 'req-0017afae-3999-4381-8db7-d47723af9b4e'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:59:39.218583 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-0017afae-3999-4381-8db7-d47723af9b4e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd00ce4a2669e43dcb5bc8a2d77b11c2f', 'request_id': 'req-0017afae-3999-4381-8db7-d47723af9b4e'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:59:39.259145 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1186/2363] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 09:59:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 09:59:39.266665 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8a78f4-0fa4-480c-81a6-353801fd8243 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:39.266944 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8a78f4-0fa4-480c-81a6-353801fd8243 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:39.267214 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8a78f4-0fa4-480c-81a6-353801fd8243 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:39.285424 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-da8a78f4-0fa4-480c-81a6-353801fd8243 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:59:39.286549 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-da8a78f4-0fa4-480c-81a6-353801fd8243 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:59:39.289974 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-0017afae-3999-4381-8db7-d47723af9b4e admin admin] The token cache is being invalidate because project d00ce4a2669e43dcb5bc8a2d77b11c2f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:59:39.300668 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-0017afae-3999-4381-8db7-d47723af9b4e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:59:39.300668 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-0017afae-3999-4381-8db7-d47723af9b4e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:59:39.300668 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1178/2364] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 24 09:59:39 2022] DELETE /identity/v3/projects/d00ce4a2669e43dcb5bc8a2d77b11c2f => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 09:59:39.300939 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1187/2365] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 09:59:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 09:59:39.307156 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8a78f4-0fa4-480c-81a6-353801fd8243 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:39.307693 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff105179-dde7-4a73-a34f-c74601885d12 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:39.307985 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff105179-dde7-4a73-a34f-c74601885d12 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:39.308264 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff105179-dde7-4a73-a34f-c74601885d12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:39.316690 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8a78f4-0fa4-480c-81a6-353801fd8243 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:39.321546 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf65baf-2784-486c-80ef-7ac7000670f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:39.322159 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf65baf-2784-486c-80ef-7ac7000670f8 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:39.322444 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf65baf-2784-486c-80ef-7ac7000670f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:39.329973 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf65baf-2784-486c-80ef-7ac7000670f8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:39.329973 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf65baf-2784-486c-80ef-7ac7000670f8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:59:39.329973 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf65baf-2784-486c-80ef-7ac7000670f8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:59:39.329973 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf65baf-2784-486c-80ef-7ac7000670f8 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:59:39.330320 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-ff105179-dde7-4a73-a34f-c74601885d12 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:59:39.330686 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-ff105179-dde7-4a73-a34f-c74601885d12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:59:39.338943 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1188/2366] 10.222.0.16 () {58 vars in 1153 bytes} [Wed Aug 24 09:59:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:59:39.339077 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1179/2367] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 09:59:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:59:39.346088 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf65baf-2784-486c-80ef-7ac7000670f8 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:39.346681 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a09727-56ac-42ea-be35-9c2a6afbdb1a None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:39.347045 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a09727-56ac-42ea-be35-9c2a6afbdb1a None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:39.347256 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a09727-56ac-42ea-be35-9c2a6afbdb1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:39.356926 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf65baf-2784-486c-80ef-7ac7000670f8 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:39.361754 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da681508-0d93-4d3c-8ee2-80687b5bb9d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:39.362400 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da681508-0d93-4d3c-8ee2-80687b5bb9d1 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:39.362683 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da681508-0d93-4d3c-8ee2-80687b5bb9d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:39.362958 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da681508-0d93-4d3c-8ee2-80687b5bb9d1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:39.363602 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da681508-0d93-4d3c-8ee2-80687b5bb9d1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:59:39.365559 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da681508-0d93-4d3c-8ee2-80687b5bb9d1 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:59:39.367101 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-b8a09727-56ac-42ea-be35-9c2a6afbdb1a None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:59:39.369093 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-b8a09727-56ac-42ea-be35-9c2a6afbdb1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:59:39.381013 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1180/2368] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 09:59:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 09:59:39.381300 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a09727-56ac-42ea-be35-9c2a6afbdb1a None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:39.391354 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a09727-56ac-42ea-be35-9c2a6afbdb1a None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:39.395987 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62773c5a-f616-4682-935f-6a4271466361 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:39.396587 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62773c5a-f616-4682-935f-6a4271466361 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:39.396861 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62773c5a-f616-4682-935f-6a4271466361 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:39.397132 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62773c5a-f616-4682-935f-6a4271466361 admin admin] PATH_INFO: `/v3/domains` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:39.397774 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62773c5a-f616-4682-935f-6a4271466361 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:59:39.398153 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62773c5a-f616-4682-935f-6a4271466361 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:59:39.400340 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62773c5a-f616-4682-935f-6a4271466361 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:59:39.415834 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1181/2369] 10.222.0.16 () {58 vars in 1153 bytes} [Wed Aug 24 09:59:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:59:39.418606 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62773c5a-f616-4682-935f-6a4271466361 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:39.428536 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62773c5a-f616-4682-935f-6a4271466361 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:39.433290 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4223c570-7f33-4c60-9c4b-42727392505e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:39.433893 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4223c570-7f33-4c60-9c4b-42727392505e admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:39.434177 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4223c570-7f33-4c60-9c4b-42727392505e admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:39.434454 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4223c570-7f33-4c60-9c4b-42727392505e admin admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:39.435077 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4223c570-7f33-4c60-9c4b-42727392505e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:59:39.437013 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4223c570-7f33-4c60-9c4b-42727392505e admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:59:39.502742 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1189/2370] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 24 09:59:39 2022] POST /identity/v3/projects => generated 367 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 09:59:39.505760 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da681508-0d93-4d3c-8ee2-80687b5bb9d1 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:39.560096 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1182/2371] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 24 09:59:39 2022] POST /identity/v3/projects => generated 385 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 09:59:39.564001 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4223c570-7f33-4c60-9c4b-42727392505e admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:39.569299 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da681508-0d93-4d3c-8ee2-80687b5bb9d1 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:39.585660 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=81,keystone:DELETE=16,keystone:INSERT=3 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:59:39.628548 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5834760e-5e11-4a81-a611-8424e0e0c82a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:39.629160 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5834760e-5e11-4a81-a611-8424e0e0c82a admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:39.629444 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5834760e-5e11-4a81-a611-8424e0e0c82a admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:39.629720 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5834760e-5e11-4a81-a611-8424e0e0c82a admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:39.630348 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5834760e-5e11-4a81-a611-8424e0e0c82a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:59:39.630567 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4223c570-7f33-4c60-9c4b-42727392505e admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:39.632388 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5834760e-5e11-4a81-a611-8424e0e0c82a admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:59:39.635325 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8648a165-5b09-45a1-b0f2-3676010376c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:39.635954 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8648a165-5b09-45a1-b0f2-3676010376c6 admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:39.636232 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8648a165-5b09-45a1-b0f2-3676010376c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:39.636502 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8648a165-5b09-45a1-b0f2-3676010376c6 admin admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:39.637152 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8648a165-5b09-45a1-b0f2-3676010376c6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:59:39.639617 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8648a165-5b09-45a1-b0f2-3676010376c6 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:59:39.723150 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1190/2372] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 24 09:59:39 2022] POST /identity/v3/users => generated 284 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 09:59:39.727879 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5834760e-5e11-4a81-a611-8424e0e0c82a admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:39.734507 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:59:39.736481 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1183/2373] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 24 09:59:39 2022] POST /identity/v3/users => generated 293 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 09:59:39.738321 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5834760e-5e11-4a81-a611-8424e0e0c82a admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:39.739987 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8648a165-5b09-45a1-b0f2-3676010376c6 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:39.743167 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3f193f-7a4e-4eae-8d6b-e336d40b1b05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:39.743792 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3f193f-7a4e-4eae-8d6b-e336d40b1b05 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:39.744113 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3f193f-7a4e-4eae-8d6b-e336d40b1b05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:39.744366 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3f193f-7a4e-4eae-8d6b-e336d40b1b05 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:39.744999 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3f193f-7a4e-4eae-8d6b-e336d40b1b05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:59:39.746606 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3f193f-7a4e-4eae-8d6b-e336d40b1b05 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:59:39.749739 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8648a165-5b09-45a1-b0f2-3676010376c6 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:39.754700 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b069ce63-d4bc-46cb-b646-356d1396f372 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:39.755299 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b069ce63-d4bc-46cb-b646-356d1396f372 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:39.755597 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b069ce63-d4bc-46cb-b646-356d1396f372 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:39.755879 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b069ce63-d4bc-46cb-b646-356d1396f372 admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:39.759577 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1191/2374] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 24 09:59:39 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 09:59:39.759733 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b069ce63-d4bc-46cb-b646-356d1396f372 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:59:39.759733 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b069ce63-d4bc-46cb-b646-356d1396f372 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:59:39.760535 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3f193f-7a4e-4eae-8d6b-e336d40b1b05 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:39.771070 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1184/2375] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 24 09:59:39 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 09:59:39.771229 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3f193f-7a4e-4eae-8d6b-e336d40b1b05 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:39.772464 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b069ce63-d4bc-46cb-b646-356d1396f372 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:39.775514 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7aeb1e-bc92-4e47-83ce-6f7f3513d911 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:39.776135 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7aeb1e-bc92-4e47-83ce-6f7f3513d911 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:39.778044 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7aeb1e-bc92-4e47-83ce-6f7f3513d911 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:39.778044 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7aeb1e-bc92-4e47-83ce-6f7f3513d911 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:39.778044 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7aeb1e-bc92-4e47-83ce-6f7f3513d911 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:59:39.778932 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7aeb1e-bc92-4e47-83ce-6f7f3513d911 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:59:39.782766 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b069ce63-d4bc-46cb-b646-356d1396f372 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:39.787903 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0162425b-e3a1-4500-85e7-2b3694028edb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:39.788495 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0162425b-e3a1-4500-85e7-2b3694028edb admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:39.788927 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0162425b-e3a1-4500-85e7-2b3694028edb admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:39.789203 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0162425b-e3a1-4500-85e7-2b3694028edb admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:39.790181 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1192/2376] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 24 09:59:39 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 09:59:39.790570 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0162425b-e3a1-4500-85e7-2b3694028edb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:59:39.792241 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0162425b-e3a1-4500-85e7-2b3694028edb admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:59:39.794245 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7aeb1e-bc92-4e47-83ce-6f7f3513d911 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:39.803699 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7aeb1e-bc92-4e47-83ce-6f7f3513d911 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:39.803837 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1185/2377] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 24 09:59:39 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 09:59:39.808756 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb45591f-e74a-4bc0-b289-e7537ef66e1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:39.809071 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0162425b-e3a1-4500-85e7-2b3694028edb admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:39.809394 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb45591f-e74a-4bc0-b289-e7537ef66e1d admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:39.809673 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb45591f-e74a-4bc0-b289-e7537ef66e1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:39.809940 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb45591f-e74a-4bc0-b289-e7537ef66e1d admin admin] PATH_INFO: `/v3/projects/5e04d6f93aff44b39f53db24f894e8bf/users/566eca13aaff4761b66e5c5a16818651/roles/136cdf78e00a426997dd4e37faff489b` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:39.818494 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0162425b-e3a1-4500-85e7-2b3694028edb admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:39.823601 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0063e4-2e1d-4758-991d-49456f69c5ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:39.824260 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0063e4-2e1d-4758-991d-49456f69c5ea admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:39.824537 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0063e4-2e1d-4758-991d-49456f69c5ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:39.824810 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0063e4-2e1d-4758-991d-49456f69c5ea admin admin] PATH_INFO: `/v3/projects/9c05e40a2edf473584e9cad9a3d52e7c/users/e70967fd97d2481195c504c5d06a0b20/roles/136cdf78e00a426997dd4e37faff489b` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:39.827312 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb45591f-e74a-4bc0-b289-e7537ef66e1d admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e04d6f93aff44b39f53db24f894e8bf, user_id=566eca13aaff4761b66e5c5a16818651, role_id=136cdf78e00a426997dd4e37faff489b)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:59:39.830120 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb45591f-e74a-4bc0-b289-e7537ef66e1d admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:59:39.840070 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0063e4-2e1d-4758-991d-49456f69c5ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c05e40a2edf473584e9cad9a3d52e7c, user_id=e70967fd97d2481195c504c5d06a0b20, role_id=136cdf78e00a426997dd4e37faff489b)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:59:39.842745 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0063e4-2e1d-4758-991d-49456f69c5ea admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:59:39.900888 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1193/2378] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 24 09:59:39 2022] PUT /identity/v3/projects/5e04d6f93aff44b39f53db24f894e8bf/users/566eca13aaff4761b66e5c5a16818651/roles/136cdf78e00a426997dd4e37faff489b => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 09:59:39.912167 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb41756-57bf-4efa-85f1-072857fce7d4 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:39.912454 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb41756-57bf-4efa-85f1-072857fce7d4 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:39.912732 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb41756-57bf-4efa-85f1-072857fce7d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:39.929991 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1186/2379] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 24 09:59:39 2022] PUT /identity/v3/projects/9c05e40a2edf473584e9cad9a3d52e7c/users/e70967fd97d2481195c504c5d06a0b20/roles/136cdf78e00a426997dd4e37faff489b => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 09:59:39.939686 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-eeb41756-57bf-4efa-85f1-072857fce7d4 None None] MFA Rules not processed for user `566eca13aaff4761b66e5c5a16818651`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:59:39.940895 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-eeb41756-57bf-4efa-85f1-072857fce7d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:59:39.947412 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfb59a3-0e48-432e-a167-556357cc44e2 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:39.947747 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfb59a3-0e48-432e-a167-556357cc44e2 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:39.948040 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfb59a3-0e48-432e-a167-556357cc44e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:39.976935 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-7cfb59a3-0e48-432e-a167-556357cc44e2 None None] MFA Rules not processed for user `e70967fd97d2481195c504c5d06a0b20`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:59:39.978144 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-7cfb59a3-0e48-432e-a167-556357cc44e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:59:40.007096 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1194/2380] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 09:59:39 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 09:59:40.018537 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb41756-57bf-4efa-85f1-072857fce7d4 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:40.020046 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-eeb41756-57bf-4efa-85f1-072857fce7d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:59:40.043402 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1187/2381] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 09:59:39 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:59:40.054533 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfb59a3-0e48-432e-a167-556357cc44e2 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:40.056569 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-7cfb59a3-0e48-432e-a167-556357cc44e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:59:40.077432 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb41756-57bf-4efa-85f1-072857fce7d4 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:40.082950 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d943a916-f873-45f6-a0ba-9fe0364ecd40 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c32b83577c46488906fe409a790b24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:40.083507 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d943a916-f873-45f6-a0ba-9fe0364ecd40 service cinder] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:40.083825 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d943a916-f873-45f6-a0ba-9fe0364ecd40 service cinder] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:40.084098 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d943a916-f873-45f6-a0ba-9fe0364ecd40 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:40.093289 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d943a916-f873-45f6-a0ba-9fe0364ecd40 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:59:40.094799 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d943a916-f873-45f6-a0ba-9fe0364ecd40 service cinder] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:59:40.104236 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1195/2382] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:59:40 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 09:59:40.115420 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:59:40.163882 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfb59a3-0e48-432e-a167-556357cc44e2 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:40.170359 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c175f75-c145-4aa5-b75e-9b061257070c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f055f7dc51d43bda9b315ca67f11d58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:40.170967 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c175f75-c145-4aa5-b75e-9b061257070c service nova] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:40.171254 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c175f75-c145-4aa5-b75e-9b061257070c service nova] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:40.171528 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c175f75-c145-4aa5-b75e-9b061257070c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:40.181155 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c175f75-c145-4aa5-b75e-9b061257070c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:59:40.182739 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c175f75-c145-4aa5-b75e-9b061257070c service nova] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:59:40.191943 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1188/2383] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:59:40 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:59:40.698025 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d943a916-f873-45f6-a0ba-9fe0364ecd40 service cinder] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:40.706807 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d943a916-f873-45f6-a0ba-9fe0364ecd40 service cinder] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:40.711273 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854544a6-1ce0-4d8b-9b36-e2da5cf692a7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c32b83577c46488906fe409a790b24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:40.711273 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854544a6-1ce0-4d8b-9b36-e2da5cf692a7 service cinder] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:40.711475 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854544a6-1ce0-4d8b-9b36-e2da5cf692a7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:40.711783 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854544a6-1ce0-4d8b-9b36-e2da5cf692a7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:40.713492 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-854544a6-1ce0-4d8b-9b36-e2da5cf692a7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:59:40.726500 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854544a6-1ce0-4d8b-9b36-e2da5cf692a7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:59:40.728529 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854544a6-1ce0-4d8b-9b36-e2da5cf692a7 service cinder] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:59:40.739273 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1196/2384] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:59:40 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:59:41.547463 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c175f75-c145-4aa5-b75e-9b061257070c service nova] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:41.558405 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c175f75-c145-4aa5-b75e-9b061257070c service nova] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:41.562046 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a15ec2-b085-4879-9858-e020f7ae20f5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c32b83577c46488906fe409a790b24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:41.562668 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a15ec2-b085-4879-9858-e020f7ae20f5 service cinder] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:41.562935 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a15ec2-b085-4879-9858-e020f7ae20f5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:41.563204 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a15ec2-b085-4879-9858-e020f7ae20f5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:41.564970 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-f9a15ec2-b085-4879-9858-e020f7ae20f5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:59:41.578113 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a15ec2-b085-4879-9858-e020f7ae20f5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:59:41.579527 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a15ec2-b085-4879-9858-e020f7ae20f5 service cinder] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:59:41.590647 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1189/2385] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 09:59:41 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:59:50.734628 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=82,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:59:51.586363 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=29 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:02:26.101324 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854544a6-1ce0-4d8b-9b36-e2da5cf692a7 service cinder] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:26.102715 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-854544a6-1ce0-4d8b-9b36-e2da5cf692a7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:02:26.169059 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854544a6-1ce0-4d8b-9b36-e2da5cf692a7 service cinder] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:26.173918 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05859eb6-3140-45b0-af1e-f02ea1ba7f90 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80cada22039d4f11935412ee578acd05', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:26.174513 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05859eb6-3140-45b0-af1e-f02ea1ba7f90 service placement] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:26.174789 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05859eb6-3140-45b0-af1e-f02ea1ba7f90 service placement] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:26.175056 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05859eb6-3140-45b0-af1e-f02ea1ba7f90 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:26.176759 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-05859eb6-3140-45b0-af1e-f02ea1ba7f90 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:02:26.187573 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05859eb6-3140-45b0-af1e-f02ea1ba7f90 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:26.189160 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05859eb6-3140-45b0-af1e-f02ea1ba7f90 service placement] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:26.199070 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1197/2386] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 10:02:26 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:02:32.842427 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a15ec2-b085-4879-9858-e020f7ae20f5 service cinder] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:32.843915 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-f9a15ec2-b085-4879-9858-e020f7ae20f5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:02:32.866227 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a15ec2-b085-4879-9858-e020f7ae20f5 service cinder] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:32.869834 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750c3ba1-9d41-4e6c-ab4a-21dad1864321 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c32b83577c46488906fe409a790b24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:32.870430 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750c3ba1-9d41-4e6c-ab4a-21dad1864321 service cinder] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:32.870716 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750c3ba1-9d41-4e6c-ab4a-21dad1864321 service cinder] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:32.871482 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750c3ba1-9d41-4e6c-ab4a-21dad1864321 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:32.873157 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-750c3ba1-9d41-4e6c-ab4a-21dad1864321 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:02:32.952216 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750c3ba1-9d41-4e6c-ab4a-21dad1864321 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:32.953783 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750c3ba1-9d41-4e6c-ab4a-21dad1864321 service cinder] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:32.962832 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1190/2387] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 10:02:32 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:02:36.194985 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=32 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:02:38.140178 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05859eb6-3140-45b0-af1e-f02ea1ba7f90 service placement] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:38.149480 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05859eb6-3140-45b0-af1e-f02ea1ba7f90 service placement] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:38.153097 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b880af-ea5a-4c17-92ec-54414d5e7615 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c32b83577c46488906fe409a790b24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:38.153714 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b880af-ea5a-4c17-92ec-54414d5e7615 service cinder] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:38.153997 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b880af-ea5a-4c17-92ec-54414d5e7615 service cinder] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:38.154269 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b880af-ea5a-4c17-92ec-54414d5e7615 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:38.156041 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-c1b880af-ea5a-4c17-92ec-54414d5e7615 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:02:38.215100 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b880af-ea5a-4c17-92ec-54414d5e7615 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:38.216566 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b880af-ea5a-4c17-92ec-54414d5e7615 service cinder] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:38.225687 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1198/2388] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 10:02:38 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:02:42.958864 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=42 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:02:48.221426 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=28 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:01.893596 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750c3ba1-9d41-4e6c-ab4a-21dad1864321 service cinder] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:01.902094 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750c3ba1-9d41-4e6c-ab4a-21dad1864321 service cinder] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:01.906572 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec678f7-a61b-40cf-8e85-b6f64f09083b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c32b83577c46488906fe409a790b24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:01.907175 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec678f7-a61b-40cf-8e85-b6f64f09083b service cinder] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:01.907453 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec678f7-a61b-40cf-8e85-b6f64f09083b service cinder] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:01.907767 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec678f7-a61b-40cf-8e85-b6f64f09083b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:01.909500 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-5ec678f7-a61b-40cf-8e85-b6f64f09083b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:01.988970 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec678f7-a61b-40cf-8e85-b6f64f09083b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:01.990408 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec678f7-a61b-40cf-8e85-b6f64f09083b service cinder] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:01.999880 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1191/2389] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 10:03:01 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:03:11.995158 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=37 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:18.483532 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b880af-ea5a-4c17-92ec-54414d5e7615 service cinder] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:18.484960 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-c1b880af-ea5a-4c17-92ec-54414d5e7615 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:18.562726 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b880af-ea5a-4c17-92ec-54414d5e7615 service cinder] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:18.566328 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf824a0-d0ca-4080-a036-9dac27ba1e90 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea2666d430a646d386104addf1f42888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:18.566933 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf824a0-d0ca-4080-a036-9dac27ba1e90 service glance] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:18.567210 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf824a0-d0ca-4080-a036-9dac27ba1e90 service glance] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:18.567481 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf824a0-d0ca-4080-a036-9dac27ba1e90 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:18.569441 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-fdf824a0-d0ca-4080-a036-9dac27ba1e90 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:18.583344 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application [None req-fdf824a0-d0ca-4080-a036-9dac27ba1e90 service glance] Could not find project: 4934cbcf5e1448fc826068c0779252b3.: keystone.exception.ProjectNotFound: Could not find project: 4934cbcf5e1448fc826068c0779252b3. Aug 24 10:03:18.583344 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 10:03:18.583344 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 10:03:18.583344 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 10:03:18.583344 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 10:03:18.583344 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 10:03:18.583344 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 10:03:18.583344 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 10:03:18.583344 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 10:03:18.583344 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 10:03:18.583344 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 10:03:18.583344 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 10:03:18.583344 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/auth.py", line 285, in get Aug 24 10:03:18.583344 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:validate_token') Aug 24 10:03:18.583344 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 422, in enforce_call Aug 24 10:03:18.584569 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application subj_token_target_data = cls._extract_subject_token_target_data() Aug 24 10:03:18.584569 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 261, in _extract_subject_token_target_data Aug 24 10:03:18.584569 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application token = PROVIDER_APIS.token_provider_api.validate_token( Aug 24 10:03:18.584569 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:03:18.584569 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:03:18.584569 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 145, in validate_token Aug 24 10:03:18.584569 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application token = self._validate_token(token_id) Aug 24 10:03:18.584569 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:03:18.584569 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:03:18.584569 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:03:18.584569 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:03:18.584569 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:03:18.584569 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 24 10:03:18.584569 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:03:18.584569 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:03:18.584569 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:03:18.584569 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:03:18.585583 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:03:18.585583 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:03:18.585583 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:03:18.585583 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:03:18.585583 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 179, in _validate_token Aug 24 10:03:18.585583 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application token.mint(token_id, issued_at) Aug 24 10:03:18.585583 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 579, in mint Aug 24 10:03:18.585583 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application self._validate_token_resources() Aug 24 10:03:18.585583 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 464, in _validate_token_resources Aug 24 10:03:18.585583 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application if self.project and not self.project.get('enabled'): Aug 24 10:03:18.585583 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 162, in project Aug 24 10:03:18.585583 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application self.__project = PROVIDERS.resource_api.get_project( Aug 24 10:03:18.585583 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 10:03:18.585583 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 10:03:18.585583 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 10:03:18.585583 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 10:03:18.585583 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 10:03:18.586506 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 10:03:18.586506 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 10:03:18.586506 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application with Lock( Aug 24 10:03:18.586506 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 10:03:18.586506 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._enter() Aug 24 10:03:18.586506 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 10:03:18.586506 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 10:03:18.586506 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 10:03:18.586506 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.creator() Aug 24 10:03:18.586506 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 10:03:18.586506 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application created_value = creator( Aug 24 10:03:18.586506 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 10:03:18.586506 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 10:03:18.586506 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 10:03:18.586506 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 10:03:18.586506 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 10:03:18.586506 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 10:03:18.587445 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: 4934cbcf5e1448fc826068c0779252b3. Aug 24 10:03:18.587445 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: ERROR keystone.server.flask.application Aug 24 10:03:18.587445 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1199/2390] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 10:03:18 2022] GET /identity/v3/auth/tokens => generated 113 bytes in 102 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Aug 24 10:03:25.250629 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec678f7-a61b-40cf-8e85-b6f64f09083b service cinder] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:25.259077 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec678f7-a61b-40cf-8e85-b6f64f09083b service cinder] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:25.262697 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9113afae-90d2-46e0-a3e3-7dbd146219bb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c32b83577c46488906fe409a790b24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:25.263292 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9113afae-90d2-46e0-a3e3-7dbd146219bb service cinder] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:25.263573 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9113afae-90d2-46e0-a3e3-7dbd146219bb service cinder] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:25.263871 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9113afae-90d2-46e0-a3e3-7dbd146219bb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:25.265614 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-9113afae-90d2-46e0-a3e3-7dbd146219bb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:25.328949 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9113afae-90d2-46e0-a3e3-7dbd146219bb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:25.330646 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9113afae-90d2-46e0-a3e3-7dbd146219bb service cinder] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:25.340289 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1192/2391] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 10:03:25 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:03:28.573426 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=33 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:33.499602 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf824a0-d0ca-4080-a036-9dac27ba1e90 service glance] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:33.501078 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-fdf824a0-d0ca-4080-a036-9dac27ba1e90 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:33.567172 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf824a0-d0ca-4080-a036-9dac27ba1e90 service glance] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:33.571956 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aba362c-198c-470a-98e8-f34eedcba365 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:33.572594 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aba362c-198c-470a-98e8-f34eedcba365 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:33.572958 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aba362c-198c-470a-98e8-f34eedcba365 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:33.573241 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aba362c-198c-470a-98e8-f34eedcba365 admin admin] PATH_INFO: `/v3/users/93a6cbe2a558474aa943682967166deb` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:33.574389 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aba362c-198c-470a-98e8-f34eedcba365 admin admin] RBAC: Authorizing `identity:delete_user(user_id=93a6cbe2a558474aa943682967166deb)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:33.576428 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aba362c-198c-470a-98e8-f34eedcba365 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:33.799419 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-1aba362c-198c-470a-98e8-f34eedcba365 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '93a6cbe2a558474aa943682967166deb'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:33.800106 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-1aba362c-198c-470a-98e8-f34eedcba365 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '93a6cbe2a558474aa943682967166deb'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:33.854269 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-1aba362c-198c-470a-98e8-f34eedcba365 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '93a6cbe2a558474aa943682967166deb'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:33.855233 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-1aba362c-198c-470a-98e8-f34eedcba365 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '93a6cbe2a558474aa943682967166deb'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:33.862688 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-1aba362c-198c-470a-98e8-f34eedcba365 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '93a6cbe2a558474aa943682967166deb'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:33.879967 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1200/2392] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 24 10:03:33 2022] DELETE /identity/v3/users/93a6cbe2a558474aa943682967166deb => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:03:33.889140 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dc8c6f-4f40-4969-a82c-dcc2b28c0e70 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:33.889140 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dc8c6f-4f40-4969-a82c-dcc2b28c0e70 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:33.889140 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dc8c6f-4f40-4969-a82c-dcc2b28c0e70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:33.906208 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-58dc8c6f-4f40-4969-a82c-dcc2b28c0e70 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:03:33.907498 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-58dc8c6f-4f40-4969-a82c-dcc2b28c0e70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:33.969654 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1193/2393] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 10:03:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:03:33.984821 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aba362c-198c-470a-98e8-f34eedcba365 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:33.986328 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-1aba362c-198c-470a-98e8-f34eedcba365 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:34.046232 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aba362c-198c-470a-98e8-f34eedcba365 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:34.049996 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d54a4a-2b52-4011-8594-ea91175e1faa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e5260e798404e8bb12b25ca8ea97052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:34.050606 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d54a4a-2b52-4011-8594-ea91175e1faa service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:34.050884 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d54a4a-2b52-4011-8594-ea91175e1faa service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:34.051154 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d54a4a-2b52-4011-8594-ea91175e1faa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:34.063868 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d54a4a-2b52-4011-8594-ea91175e1faa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:34.065278 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d54a4a-2b52-4011-8594-ea91175e1faa service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:34.074761 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1201/2394] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 10:03:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:03:34.529716 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dc8c6f-4f40-4969-a82c-dcc2b28c0e70 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:34.534457 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-58dc8c6f-4f40-4969-a82c-dcc2b28c0e70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:34.558240 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dc8c6f-4f40-4969-a82c-dcc2b28c0e70 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:34.586257 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80a00d1-a63b-4778-b36f-7865f34f31bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:34.587470 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80a00d1-a63b-4778-b36f-7865f34f31bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:34.587978 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80a00d1-a63b-4778-b36f-7865f34f31bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:34.588443 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80a00d1-a63b-4778-b36f-7865f34f31bf admin admin] PATH_INFO: `/v3/projects/885df5002d584d3e851b890198f21550` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:34.590942 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80a00d1-a63b-4778-b36f-7865f34f31bf admin admin] RBAC: Authorizing `identity:delete_project(project_id=885df5002d584d3e851b890198f21550)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:34.593577 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80a00d1-a63b-4778-b36f-7865f34f31bf admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:34.715272 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d54a4a-2b52-4011-8594-ea91175e1faa service neutron] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:34.719134 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-b5d54a4a-2b52-4011-8594-ea91175e1faa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:34.733519 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-d80a00d1-a63b-4778-b36f-7865f34f31bf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '885df5002d584d3e851b890198f21550', 'request_id': 'req-d80a00d1-a63b-4778-b36f-7865f34f31bf'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:34.738682 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d54a4a-2b52-4011-8594-ea91175e1faa service neutron] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:34.740331 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-d80a00d1-a63b-4778-b36f-7865f34f31bf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '885df5002d584d3e851b890198f21550', 'request_id': 'req-d80a00d1-a63b-4778-b36f-7865f34f31bf'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:34.746770 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-d80a00d1-a63b-4778-b36f-7865f34f31bf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '885df5002d584d3e851b890198f21550', 'request_id': 'req-d80a00d1-a63b-4778-b36f-7865f34f31bf'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:34.760855 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:34.805073 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379fdbec-a5fd-462e-809d-2444a75fef0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:34.806293 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379fdbec-a5fd-462e-809d-2444a75fef0a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:34.806901 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379fdbec-a5fd-462e-809d-2444a75fef0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:34.807366 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379fdbec-a5fd-462e-809d-2444a75fef0a admin admin] PATH_INFO: `/v3/users/e70967fd97d2481195c504c5d06a0b20` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:34.808847 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379fdbec-a5fd-462e-809d-2444a75fef0a admin admin] RBAC: Authorizing `identity:delete_user(user_id=e70967fd97d2481195c504c5d06a0b20)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:34.820110 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379fdbec-a5fd-462e-809d-2444a75fef0a admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:34.835619 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-d80a00d1-a63b-4778-b36f-7865f34f31bf admin admin] The token cache is being invalidate because project 885df5002d584d3e851b890198f21550 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:03:34.845388 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-d80a00d1-a63b-4778-b36f-7865f34f31bf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:34.845388 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-d80a00d1-a63b-4778-b36f-7865f34f31bf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:34.845388 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1194/2395] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 24 10:03:34 2022] DELETE /identity/v3/projects/885df5002d584d3e851b890198f21550 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:03:34.848944 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80a00d1-a63b-4778-b36f-7865f34f31bf admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:34.850690 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-d80a00d1-a63b-4778-b36f-7865f34f31bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:34.864477 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80a00d1-a63b-4778-b36f-7865f34f31bf admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:34.869577 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e0ce21-bcca-46ea-b00d-fa1b764c250d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:34.870191 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e0ce21-bcca-46ea-b00d-fa1b764c250d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:34.870473 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e0ce21-bcca-46ea-b00d-fa1b764c250d admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:34.870747 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e0ce21-bcca-46ea-b00d-fa1b764c250d admin admin] PATH_INFO: `/v3/users/ca9d815de9e54796ae265d53e528b239` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:34.871922 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e0ce21-bcca-46ea-b00d-fa1b764c250d admin admin] RBAC: Authorizing `identity:delete_user(user_id=ca9d815de9e54796ae265d53e528b239)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:34.874173 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e0ce21-bcca-46ea-b00d-fa1b764c250d admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:34.892543 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:34.983826 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-379fdbec-a5fd-462e-809d-2444a75fef0a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e70967fd97d2481195c504c5d06a0b20'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:34.984479 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-379fdbec-a5fd-462e-809d-2444a75fef0a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e70967fd97d2481195c504c5d06a0b20'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:35.061303 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-379fdbec-a5fd-462e-809d-2444a75fef0a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e70967fd97d2481195c504c5d06a0b20'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:35.062523 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-379fdbec-a5fd-462e-809d-2444a75fef0a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e70967fd97d2481195c504c5d06a0b20'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:35.071598 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-379fdbec-a5fd-462e-809d-2444a75fef0a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e70967fd97d2481195c504c5d06a0b20'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:35.076111 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-30e0ce21-bcca-46ea-b00d-fa1b764c250d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ca9d815de9e54796ae265d53e528b239'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:35.076797 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-30e0ce21-bcca-46ea-b00d-fa1b764c250d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ca9d815de9e54796ae265d53e528b239'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:35.090873 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-30e0ce21-bcca-46ea-b00d-fa1b764c250d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ca9d815de9e54796ae265d53e528b239'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:35.091402 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1202/2396] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 24 10:03:34 2022] DELETE /identity/v3/users/e70967fd97d2481195c504c5d06a0b20 => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:03:35.095564 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77abd735-9336-4bdb-9a93-677ed99d10ff None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:35.095882 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77abd735-9336-4bdb-9a93-677ed99d10ff None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:35.096182 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77abd735-9336-4bdb-9a93-677ed99d10ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:35.098057 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-30e0ce21-bcca-46ea-b00d-fa1b764c250d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ca9d815de9e54796ae265d53e528b239'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:35.114782 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-77abd735-9336-4bdb-9a93-677ed99d10ff None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:03:35.116045 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-77abd735-9336-4bdb-9a93-677ed99d10ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:35.152717 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-30e0ce21-bcca-46ea-b00d-fa1b764c250d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ca9d815de9e54796ae265d53e528b239'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:35.159109 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1195/2397] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 24 10:03:34 2022] DELETE /identity/v3/users/ca9d815de9e54796ae265d53e528b239 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:03:35.174509 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1203/2398] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 10:03:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:03:35.184984 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e0ce21-bcca-46ea-b00d-fa1b764c250d admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:35.187024 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-30e0ce21-bcca-46ea-b00d-fa1b764c250d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:35.245581 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e0ce21-bcca-46ea-b00d-fa1b764c250d admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:35.249971 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9996e570-2457-4cc8-93f9-dc64aebfec76 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e5260e798404e8bb12b25ca8ea97052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:35.250583 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9996e570-2457-4cc8-93f9-dc64aebfec76 service neutron] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:35.251407 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9996e570-2457-4cc8-93f9-dc64aebfec76 service neutron] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:35.251740 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9996e570-2457-4cc8-93f9-dc64aebfec76 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:35.313064 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9996e570-2457-4cc8-93f9-dc64aebfec76 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:35.314434 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9996e570-2457-4cc8-93f9-dc64aebfec76 service neutron] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:35.324046 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1196/2399] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 10:03:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:03:35.608233 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77abd735-9336-4bdb-9a93-677ed99d10ff None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:35.611248 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-77abd735-9336-4bdb-9a93-677ed99d10ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:35.632291 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77abd735-9336-4bdb-9a93-677ed99d10ff None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:35.639217 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff034fc-9568-4913-85b4-43cc361a2f82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:35.640738 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff034fc-9568-4913-85b4-43cc361a2f82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:35.641075 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff034fc-9568-4913-85b4-43cc361a2f82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:35.641297 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff034fc-9568-4913-85b4-43cc361a2f82 admin admin] PATH_INFO: `/v3/projects/f6ea09160bff441aa713ab604e219575` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:35.643090 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff034fc-9568-4913-85b4-43cc361a2f82 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f6ea09160bff441aa713ab604e219575)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:35.646248 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff034fc-9568-4913-85b4-43cc361a2f82 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:35.754725 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9996e570-2457-4cc8-93f9-dc64aebfec76 service neutron] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:35.756242 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-9996e570-2457-4cc8-93f9-dc64aebfec76 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:35.759277 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:35.768733 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-cff034fc-9568-4913-85b4-43cc361a2f82 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f6ea09160bff441aa713ab604e219575', 'request_id': 'req-cff034fc-9568-4913-85b4-43cc361a2f82'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:35.770280 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9996e570-2457-4cc8-93f9-dc64aebfec76 service neutron] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:35.776581 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-cff034fc-9568-4913-85b4-43cc361a2f82 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f6ea09160bff441aa713ab604e219575', 'request_id': 'req-cff034fc-9568-4913-85b4-43cc361a2f82'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:35.785992 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-cff034fc-9568-4913-85b4-43cc361a2f82 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f6ea09160bff441aa713ab604e219575', 'request_id': 'req-cff034fc-9568-4913-85b4-43cc361a2f82'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:35.812487 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:35.830569 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397115d4-4c4b-441c-a883-d1ab8a6c1297 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:35.831298 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397115d4-4c4b-441c-a883-d1ab8a6c1297 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:35.831514 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397115d4-4c4b-441c-a883-d1ab8a6c1297 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:35.831833 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397115d4-4c4b-441c-a883-d1ab8a6c1297 admin admin] PATH_INFO: `/v3/projects/9c05e40a2edf473584e9cad9a3d52e7c` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:35.832969 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397115d4-4c4b-441c-a883-d1ab8a6c1297 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9c05e40a2edf473584e9cad9a3d52e7c)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:35.834902 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397115d4-4c4b-441c-a883-d1ab8a6c1297 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:35.837602 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-cff034fc-9568-4913-85b4-43cc361a2f82 admin admin] The token cache is being invalidate because project f6ea09160bff441aa713ab604e219575 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81602) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:03:35.837950 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-cff034fc-9568-4913-85b4-43cc361a2f82 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:35.838584 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-cff034fc-9568-4913-85b4-43cc361a2f82 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:35.840121 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1204/2400] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 24 10:03:35 2022] DELETE /identity/v3/projects/f6ea09160bff441aa713ab604e219575 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:03:35.845399 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e879f79-37aa-4a79-a2eb-6b486cc30dfb None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:35.845812 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e879f79-37aa-4a79-a2eb-6b486cc30dfb None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:35.846095 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e879f79-37aa-4a79-a2eb-6b486cc30dfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:35.866804 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-4e879f79-37aa-4a79-a2eb-6b486cc30dfb None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:03:35.868079 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-4e879f79-37aa-4a79-a2eb-6b486cc30dfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:35.876080 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1205/2401] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 10:03:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:03:35.883576 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2952cb5b-ba3d-471c-a8a4-8195a8fba911 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:35.884126 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2952cb5b-ba3d-471c-a8a4-8195a8fba911 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:35.884464 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2952cb5b-ba3d-471c-a8a4-8195a8fba911 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:35.904040 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-2952cb5b-ba3d-471c-a8a4-8195a8fba911 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:03:35.905802 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-2952cb5b-ba3d-471c-a8a4-8195a8fba911 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:35.915320 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1206/2402] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 10:03:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:03:35.919574 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2952cb5b-ba3d-471c-a8a4-8195a8fba911 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:35.929316 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2952cb5b-ba3d-471c-a8a4-8195a8fba911 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:35.937913 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff19942-8d44-40a5-b643-e166b0aa4d09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:35.941123 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff19942-8d44-40a5-b643-e166b0aa4d09 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:35.941123 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff19942-8d44-40a5-b643-e166b0aa4d09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:35.941123 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff19942-8d44-40a5-b643-e166b0aa4d09 admin admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:35.941123 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff19942-8d44-40a5-b643-e166b0aa4d09 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:03:35.941123 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff19942-8d44-40a5-b643-e166b0aa4d09 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:35.942309 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff19942-8d44-40a5-b643-e166b0aa4d09 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:35.955049 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1207/2403] 10.222.0.16 () {58 vars in 1153 bytes} [Wed Aug 24 10:03:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 10:03:35.960270 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff19942-8d44-40a5-b643-e166b0aa4d09 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:35.970777 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-397115d4-4c4b-441c-a883-d1ab8a6c1297 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9c05e40a2edf473584e9cad9a3d52e7c', 'request_id': 'req-397115d4-4c4b-441c-a883-d1ab8a6c1297'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:35.971243 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff19942-8d44-40a5-b643-e166b0aa4d09 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:35.977971 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-397115d4-4c4b-441c-a883-d1ab8a6c1297 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9c05e40a2edf473584e9cad9a3d52e7c', 'request_id': 'req-397115d4-4c4b-441c-a883-d1ab8a6c1297'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:35.984913 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-397115d4-4c4b-441c-a883-d1ab8a6c1297 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9c05e40a2edf473584e9cad9a3d52e7c', 'request_id': 'req-397115d4-4c4b-441c-a883-d1ab8a6c1297'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:36.029277 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eabcddc-7ac3-46c7-9553-cf423e2bfc39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:36.029889 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eabcddc-7ac3-46c7-9553-cf423e2bfc39 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:36.030171 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eabcddc-7ac3-46c7-9553-cf423e2bfc39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:36.031137 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eabcddc-7ac3-46c7-9553-cf423e2bfc39 admin admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:36.031824 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eabcddc-7ac3-46c7-9553-cf423e2bfc39 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:36.033719 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eabcddc-7ac3-46c7-9553-cf423e2bfc39 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:36.049457 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-397115d4-4c4b-441c-a883-d1ab8a6c1297 admin admin] The token cache is being invalidate because project 9c05e40a2edf473584e9cad9a3d52e7c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:03:36.049729 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-397115d4-4c4b-441c-a883-d1ab8a6c1297 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:36.050422 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-397115d4-4c4b-441c-a883-d1ab8a6c1297 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:36.052555 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1197/2404] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 24 10:03:35 2022] DELETE /identity/v3/projects/9c05e40a2edf473584e9cad9a3d52e7c => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:03:36.058109 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9b236d-94d4-457f-b872-fdf9938e0955 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:36.058416 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9b236d-94d4-457f-b872-fdf9938e0955 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:36.058697 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9b236d-94d4-457f-b872-fdf9938e0955 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:36.079040 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-1c9b236d-94d4-457f-b872-fdf9938e0955 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:03:36.081120 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-1c9b236d-94d4-457f-b872-fdf9938e0955 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:36.089107 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1198/2405] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 10:03:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:03:36.098612 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b1e890-9780-4422-8e6b-6751355e411b None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:36.098892 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b1e890-9780-4422-8e6b-6751355e411b None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:36.099166 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b1e890-9780-4422-8e6b-6751355e411b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:36.123701 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-46b1e890-9780-4422-8e6b-6751355e411b None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:03:36.123860 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1208/2406] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 24 10:03:35 2022] POST /identity/v3/projects => generated 387 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:03:36.124030 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-46b1e890-9780-4422-8e6b-6751355e411b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:36.124381 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eabcddc-7ac3-46c7-9553-cf423e2bfc39 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:36.126008 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-9eabcddc-7ac3-46c7-9553-cf423e2bfc39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:36.184273 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1199/2407] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 10:03:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:03:36.188436 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b1e890-9780-4422-8e6b-6751355e411b None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:36.190016 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eabcddc-7ac3-46c7-9553-cf423e2bfc39 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:36.194764 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a62331-6b1f-40dc-9c5f-c7d8b04c0b52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:36.195378 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a62331-6b1f-40dc-9c5f-c7d8b04c0b52 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:36.195694 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a62331-6b1f-40dc-9c5f-c7d8b04c0b52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:36.195984 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a62331-6b1f-40dc-9c5f-c7d8b04c0b52 admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:36.196607 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a62331-6b1f-40dc-9c5f-c7d8b04c0b52 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:36.197209 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b1e890-9780-4422-8e6b-6751355e411b None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:36.198562 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a62331-6b1f-40dc-9c5f-c7d8b04c0b52 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:36.202683 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc94b20-190f-4138-b7c9-906b0e8ab4a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:36.203328 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc94b20-190f-4138-b7c9-906b0e8ab4a1 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:36.203647 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc94b20-190f-4138-b7c9-906b0e8ab4a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:36.204590 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc94b20-190f-4138-b7c9-906b0e8ab4a1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:36.204886 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc94b20-190f-4138-b7c9-906b0e8ab4a1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:03:36.205191 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc94b20-190f-4138-b7c9-906b0e8ab4a1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:36.206776 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc94b20-190f-4138-b7c9-906b0e8ab4a1 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:36.220195 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1200/2408] 10.222.0.16 () {58 vars in 1153 bytes} [Wed Aug 24 10:03:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 10:03:36.224595 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc94b20-190f-4138-b7c9-906b0e8ab4a1 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:36.235519 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc94b20-190f-4138-b7c9-906b0e8ab4a1 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:36.240426 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b11d2c-97e5-4171-942e-b716d39528c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:36.241034 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b11d2c-97e5-4171-942e-b716d39528c5 admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:36.241316 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b11d2c-97e5-4171-942e-b716d39528c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:36.241592 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b11d2c-97e5-4171-942e-b716d39528c5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:36.242215 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b11d2c-97e5-4171-942e-b716d39528c5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:36.244211 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b11d2c-97e5-4171-942e-b716d39528c5 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:36.313018 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1209/2409] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 24 10:03:36 2022] POST /identity/v3/users => generated 294 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 10:03:36.317068 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a62331-6b1f-40dc-9c5f-c7d8b04c0b52 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:36.326525 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a62331-6b1f-40dc-9c5f-c7d8b04c0b52 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:36.332119 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909f2f5a-c9da-4068-b9e7-1d4726f8fa9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:36.332722 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909f2f5a-c9da-4068-b9e7-1d4726f8fa9e admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:36.333005 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909f2f5a-c9da-4068-b9e7-1d4726f8fa9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:36.333275 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909f2f5a-c9da-4068-b9e7-1d4726f8fa9e admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:36.334265 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909f2f5a-c9da-4068-b9e7-1d4726f8fa9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:36.335960 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909f2f5a-c9da-4068-b9e7-1d4726f8fa9e admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:36.347720 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1201/2410] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 24 10:03:36 2022] POST /identity/v3/projects => generated 383 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 10:03:36.347885 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1210/2411] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 24 10:03:36 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 10:03:36.349189 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b11d2c-97e5-4171-942e-b716d39528c5 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:36.351446 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909f2f5a-c9da-4068-b9e7-1d4726f8fa9e admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:36.356445 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:36.362129 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909f2f5a-c9da-4068-b9e7-1d4726f8fa9e admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:36.380511 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:36.411841 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b11d2c-97e5-4171-942e-b716d39528c5 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:36.416742 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81928105-ce21-4602-9e9e-4eda1836526a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:36.417407 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81928105-ce21-4602-9e9e-4eda1836526a admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:36.417686 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81928105-ce21-4602-9e9e-4eda1836526a admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:36.417959 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81928105-ce21-4602-9e9e-4eda1836526a admin admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:36.418635 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81928105-ce21-4602-9e9e-4eda1836526a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:36.419854 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dc9250-0a63-437b-882b-40ebfd7955eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:36.421627 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dc9250-0a63-437b-882b-40ebfd7955eb admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:36.421777 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81928105-ce21-4602-9e9e-4eda1836526a admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:36.421914 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dc9250-0a63-437b-882b-40ebfd7955eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:36.421914 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dc9250-0a63-437b-882b-40ebfd7955eb admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:36.421914 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29dc9250-0a63-437b-882b-40ebfd7955eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:36.423474 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29dc9250-0a63-437b-882b-40ebfd7955eb admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:36.433463 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1211/2412] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 24 10:03:36 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 10:03:36.438490 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dc9250-0a63-437b-882b-40ebfd7955eb admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:36.448119 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dc9250-0a63-437b-882b-40ebfd7955eb admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:36.453260 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96ce93d-1a23-4988-aed6-307200bc8300 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:36.453884 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96ce93d-1a23-4988-aed6-307200bc8300 admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:36.454158 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96ce93d-1a23-4988-aed6-307200bc8300 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:36.454684 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96ce93d-1a23-4988-aed6-307200bc8300 admin admin] PATH_INFO: `/v3/projects/37bf221d7438497e9ea138f0c18d1e35/users/bfec0f7f68b84818a24c939f250fc067/roles/136cdf78e00a426997dd4e37faff489b` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:36.469522 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96ce93d-1a23-4988-aed6-307200bc8300 admin admin] RBAC: Authorizing `identity:create_grant(project_id=37bf221d7438497e9ea138f0c18d1e35, user_id=bfec0f7f68b84818a24c939f250fc067, role_id=136cdf78e00a426997dd4e37faff489b)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:36.472254 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96ce93d-1a23-4988-aed6-307200bc8300 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:36.490352 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1202/2413] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 24 10:03:36 2022] POST /identity/v3/users => generated 292 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 10:03:36.494086 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81928105-ce21-4602-9e9e-4eda1836526a admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:36.510102 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81928105-ce21-4602-9e9e-4eda1836526a admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:36.515371 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e779d7db-35fc-4f8e-b2d3-c0350818f2aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:36.516017 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e779d7db-35fc-4f8e-b2d3-c0350818f2aa admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:36.516418 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e779d7db-35fc-4f8e-b2d3-c0350818f2aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:36.516796 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e779d7db-35fc-4f8e-b2d3-c0350818f2aa admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:36.517336 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e779d7db-35fc-4f8e-b2d3-c0350818f2aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:36.518919 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e779d7db-35fc-4f8e-b2d3-c0350818f2aa admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:36.528627 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1203/2414] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 24 10:03:36 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 10:03:36.532460 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e779d7db-35fc-4f8e-b2d3-c0350818f2aa admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:36.542295 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e779d7db-35fc-4f8e-b2d3-c0350818f2aa admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:36.547346 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1212/2415] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 24 10:03:36 2022] PUT /identity/v3/projects/37bf221d7438497e9ea138f0c18d1e35/users/bfec0f7f68b84818a24c939f250fc067/roles/136cdf78e00a426997dd4e37faff489b => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 10:03:36.567691 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dde07e-1c0a-4b6e-a4f2-4e70a5a9ecd2 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:36.567977 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dde07e-1c0a-4b6e-a4f2-4e70a5a9ecd2 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:36.568251 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dde07e-1c0a-4b6e-a4f2-4e70a5a9ecd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:36.596643 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-21dde07e-1c0a-4b6e-a4f2-4e70a5a9ecd2 None None] MFA Rules not processed for user `bfec0f7f68b84818a24c939f250fc067`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:03:36.598158 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-21dde07e-1c0a-4b6e-a4f2-4e70a5a9ecd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:36.601250 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c9b2cc-95af-41e9-8726-05b71829376a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:36.601983 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c9b2cc-95af-41e9-8726-05b71829376a admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:36.602262 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c9b2cc-95af-41e9-8726-05b71829376a admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:36.602597 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c9b2cc-95af-41e9-8726-05b71829376a admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:36.603243 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c9b2cc-95af-41e9-8726-05b71829376a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:36.604963 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c9b2cc-95af-41e9-8726-05b71829376a admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:36.618342 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1204/2416] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 24 10:03:36 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 10:03:36.619395 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c9b2cc-95af-41e9-8726-05b71829376a admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:36.629646 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c9b2cc-95af-41e9-8726-05b71829376a admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:36.635687 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18254f65-ebc6-43d8-a4e9-61dc7285e6a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:36.636525 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18254f65-ebc6-43d8-a4e9-61dc7285e6a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:36.636766 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18254f65-ebc6-43d8-a4e9-61dc7285e6a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:36.637040 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18254f65-ebc6-43d8-a4e9-61dc7285e6a4 admin admin] PATH_INFO: `/v3/projects/0efa2cdf9808421682838cfb83e31a38/users/b3c74acd731547078091f85d1c769ea3/roles/136cdf78e00a426997dd4e37faff489b` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:36.652565 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18254f65-ebc6-43d8-a4e9-61dc7285e6a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0efa2cdf9808421682838cfb83e31a38, user_id=b3c74acd731547078091f85d1c769ea3, role_id=136cdf78e00a426997dd4e37faff489b)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:36.656028 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18254f65-ebc6-43d8-a4e9-61dc7285e6a4 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:36.669049 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1213/2417] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 10:03:36 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:03:36.674137 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dde07e-1c0a-4b6e-a4f2-4e70a5a9ecd2 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:36.684294 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dde07e-1c0a-4b6e-a4f2-4e70a5a9ecd2 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:36.689099 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248bcbb9-a71c-4a72-bdfa-0ba3078126a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:36.689683 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248bcbb9-a71c-4a72-bdfa-0ba3078126a8 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:36.689964 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248bcbb9-a71c-4a72-bdfa-0ba3078126a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:36.690234 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248bcbb9-a71c-4a72-bdfa-0ba3078126a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:36.690846 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248bcbb9-a71c-4a72-bdfa-0ba3078126a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:36.692863 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248bcbb9-a71c-4a72-bdfa-0ba3078126a8 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:36.746619 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1205/2418] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 24 10:03:36 2022] PUT /identity/v3/projects/0efa2cdf9808421682838cfb83e31a38/users/b3c74acd731547078091f85d1c769ea3/roles/136cdf78e00a426997dd4e37faff489b => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:03:36.756740 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953510fc-950c-4a6a-adca-2f0d31fecac9 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:36.757150 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953510fc-950c-4a6a-adca-2f0d31fecac9 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:36.757454 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953510fc-950c-4a6a-adca-2f0d31fecac9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:36.785327 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-953510fc-950c-4a6a-adca-2f0d31fecac9 None None] MFA Rules not processed for user `b3c74acd731547078091f85d1c769ea3`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:03:36.786547 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-953510fc-950c-4a6a-adca-2f0d31fecac9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:36.804337 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1214/2419] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 24 10:03:36 2022] POST /identity/v3/projects => generated 387 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 10:03:36.808137 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248bcbb9-a71c-4a72-bdfa-0ba3078126a8 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:36.854815 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:36.856881 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1206/2420] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 10:03:36 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:03:36.861464 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953510fc-950c-4a6a-adca-2f0d31fecac9 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:36.871553 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:36.871864 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248bcbb9-a71c-4a72-bdfa-0ba3078126a8 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:36.876829 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8110d96f-13aa-418a-afb6-955a74222139 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:36.877449 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8110d96f-13aa-418a-afb6-955a74222139 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:36.877732 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8110d96f-13aa-418a-afb6-955a74222139 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:36.878003 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8110d96f-13aa-418a-afb6-955a74222139 admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:36.878649 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8110d96f-13aa-418a-afb6-955a74222139 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:36.880684 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8110d96f-13aa-418a-afb6-955a74222139 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:36.923845 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953510fc-950c-4a6a-adca-2f0d31fecac9 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:36.928467 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0605d2-f4cd-4ddf-9e41-5d500e8055df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:36.937253 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0605d2-f4cd-4ddf-9e41-5d500e8055df admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:36.937253 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0605d2-f4cd-4ddf-9e41-5d500e8055df admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:36.937253 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0605d2-f4cd-4ddf-9e41-5d500e8055df admin admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:36.937253 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0605d2-f4cd-4ddf-9e41-5d500e8055df admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:36.937253 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0605d2-f4cd-4ddf-9e41-5d500e8055df admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:36.960977 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1215/2421] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 24 10:03:36 2022] POST /identity/v3/users => generated 294 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 10:03:36.964609 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8110d96f-13aa-418a-afb6-955a74222139 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:36.975452 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8110d96f-13aa-418a-afb6-955a74222139 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:36.980314 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd9ec87-a09a-463a-9e2a-be067338bbf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:36.980927 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd9ec87-a09a-463a-9e2a-be067338bbf5 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:36.985974 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd9ec87-a09a-463a-9e2a-be067338bbf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:36.985974 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd9ec87-a09a-463a-9e2a-be067338bbf5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:36.985974 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd9ec87-a09a-463a-9e2a-be067338bbf5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:36.985974 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd9ec87-a09a-463a-9e2a-be067338bbf5 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:36.994480 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1216/2422] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 24 10:03:36 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:03:36.998304 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd9ec87-a09a-463a-9e2a-be067338bbf5 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:37.017152 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1207/2423] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 24 10:03:36 2022] POST /identity/v3/projects => generated 387 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 10:03:37.019253 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0605d2-f4cd-4ddf-9e41-5d500e8055df admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:37.024701 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd9ec87-a09a-463a-9e2a-be067338bbf5 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:37.092141 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009e46a5-a75b-440a-b2a8-d71642990721 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:37.093338 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009e46a5-a75b-440a-b2a8-d71642990721 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:37.093633 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009e46a5-a75b-440a-b2a8-d71642990721 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:37.094008 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009e46a5-a75b-440a-b2a8-d71642990721 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:37.095079 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009e46a5-a75b-440a-b2a8-d71642990721 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:37.096960 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009e46a5-a75b-440a-b2a8-d71642990721 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:37.107146 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0605d2-f4cd-4ddf-9e41-5d500e8055df admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:37.108454 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1217/2424] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 24 10:03:36 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 10:03:37.112858 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009e46a5-a75b-440a-b2a8-d71642990721 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:37.113037 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ab9c2e-91d2-4cae-81aa-d42e308c722f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:37.113634 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ab9c2e-91d2-4cae-81aa-d42e308c722f admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:37.113920 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ab9c2e-91d2-4cae-81aa-d42e308c722f admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:37.114198 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ab9c2e-91d2-4cae-81aa-d42e308c722f admin admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:37.114828 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ab9c2e-91d2-4cae-81aa-d42e308c722f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:37.116791 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ab9c2e-91d2-4cae-81aa-d42e308c722f admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:37.125106 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009e46a5-a75b-440a-b2a8-d71642990721 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:37.135465 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e73c56-3860-472f-888b-69e947810476 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:37.136277 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e73c56-3860-472f-888b-69e947810476 admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:37.136568 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e73c56-3860-472f-888b-69e947810476 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:37.136831 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e73c56-3860-472f-888b-69e947810476 admin admin] PATH_INFO: `/v3/projects/b3808bba24664822a72296a290f5ceef/users/36546304609647538d464396ed4e8496/roles/136cdf78e00a426997dd4e37faff489b` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:37.152718 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e73c56-3860-472f-888b-69e947810476 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3808bba24664822a72296a290f5ceef, user_id=36546304609647538d464396ed4e8496, role_id=136cdf78e00a426997dd4e37faff489b)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:37.155956 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e73c56-3860-472f-888b-69e947810476 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:37.193648 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1208/2425] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 24 10:03:37 2022] POST /identity/v3/users => generated 300 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 10:03:37.197428 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ab9c2e-91d2-4cae-81aa-d42e308c722f admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:37.207928 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ab9c2e-91d2-4cae-81aa-d42e308c722f admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:37.215690 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1218/2426] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 24 10:03:37 2022] PUT /identity/v3/projects/b3808bba24664822a72296a290f5ceef/users/36546304609647538d464396ed4e8496/roles/136cdf78e00a426997dd4e37faff489b => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 10:03:37.230323 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbf9740-4fc5-4ffb-9154-18ca06c14b39 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:37.230606 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbf9740-4fc5-4ffb-9154-18ca06c14b39 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:37.230881 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbf9740-4fc5-4ffb-9154-18ca06c14b39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:37.259444 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-8fbf9740-4fc5-4ffb-9154-18ca06c14b39 None None] MFA Rules not processed for user `36546304609647538d464396ed4e8496`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:03:37.260965 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-8fbf9740-4fc5-4ffb-9154-18ca06c14b39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:37.270734 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e363a5f-a044-4ac4-84b6-94ab46c89eb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:37.271451 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e363a5f-a044-4ac4-84b6-94ab46c89eb4 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:37.271767 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e363a5f-a044-4ac4-84b6-94ab46c89eb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:37.272044 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e363a5f-a044-4ac4-84b6-94ab46c89eb4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:37.272694 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e363a5f-a044-4ac4-84b6-94ab46c89eb4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:37.274306 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e363a5f-a044-4ac4-84b6-94ab46c89eb4 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:37.287727 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1209/2427] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 24 10:03:37 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 10:03:37.289306 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e363a5f-a044-4ac4-84b6-94ab46c89eb4 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:37.302324 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e363a5f-a044-4ac4-84b6-94ab46c89eb4 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:37.309376 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c333da-696f-488e-a6c5-ffc7c03f5577 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:37.310952 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c333da-696f-488e-a6c5-ffc7c03f5577 admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:37.311925 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c333da-696f-488e-a6c5-ffc7c03f5577 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:37.312225 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c333da-696f-488e-a6c5-ffc7c03f5577 admin admin] PATH_INFO: `/v3/projects/4f392d7e74404c218bab9b5b44d17abc/users/90ed62fc171b47298023cb7601e08b59/roles/15f634d4a2d247ddab90c458e5e1c141` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:37.341570 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1219/2428] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 10:03:37 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:03:37.345940 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbf9740-4fc5-4ffb-9154-18ca06c14b39 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:37.346350 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c333da-696f-488e-a6c5-ffc7c03f5577 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f392d7e74404c218bab9b5b44d17abc, user_id=90ed62fc171b47298023cb7601e08b59, role_id=15f634d4a2d247ddab90c458e5e1c141)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:37.350880 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c333da-696f-488e-a6c5-ffc7c03f5577 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:37.354937 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:37.357793 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbf9740-4fc5-4ffb-9154-18ca06c14b39 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:37.363118 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88dfbcf5-7f67-4072-baca-58c77f99a96f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:37.363835 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dfbcf5-7f67-4072-baca-58c77f99a96f admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:37.364178 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dfbcf5-7f67-4072-baca-58c77f99a96f admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:37.364457 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dfbcf5-7f67-4072-baca-58c77f99a96f admin admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:37.365114 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88dfbcf5-7f67-4072-baca-58c77f99a96f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:37.367005 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88dfbcf5-7f67-4072-baca-58c77f99a96f admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:37.407499 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1210/2429] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 24 10:03:37 2022] PUT /identity/v3/projects/4f392d7e74404c218bab9b5b44d17abc/users/90ed62fc171b47298023cb7601e08b59/roles/15f634d4a2d247ddab90c458e5e1c141 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:03:37.411239 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c333da-696f-488e-a6c5-ffc7c03f5577 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:37.472279 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c333da-696f-488e-a6c5-ffc7c03f5577 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:37.477085 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17b5f52-392f-47ae-b32d-4b1c32bc29ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:37.477831 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17b5f52-392f-47ae-b32d-4b1c32bc29ae admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:37.478112 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17b5f52-392f-47ae-b32d-4b1c32bc29ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:37.478403 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17b5f52-392f-47ae-b32d-4b1c32bc29ae admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:37.479051 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17b5f52-392f-47ae-b32d-4b1c32bc29ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:37.480714 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17b5f52-392f-47ae-b32d-4b1c32bc29ae admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:37.485370 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:37.494504 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1211/2430] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 24 10:03:37 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 10:03:37.494663 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1220/2431] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 24 10:03:37 2022] POST /identity/v3/projects => generated 387 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 10:03:37.496643 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17b5f52-392f-47ae-b32d-4b1c32bc29ae admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:37.497182 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88dfbcf5-7f67-4072-baca-58c77f99a96f admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:37.507407 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17b5f52-392f-47ae-b32d-4b1c32bc29ae admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:37.559488 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88dfbcf5-7f67-4072-baca-58c77f99a96f admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:37.565095 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30646782-b1f2-4c63-be51-17dd2e9d07cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:37.565857 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30646782-b1f2-4c63-be51-17dd2e9d07cd admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:37.565971 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ac36b5-1400-4a10-a94a-c1cafb4b5846 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:37.566141 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30646782-b1f2-4c63-be51-17dd2e9d07cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:37.566417 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30646782-b1f2-4c63-be51-17dd2e9d07cd admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:37.566607 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ac36b5-1400-4a10-a94a-c1cafb4b5846 admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:37.566885 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ac36b5-1400-4a10-a94a-c1cafb4b5846 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:37.567039 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30646782-b1f2-4c63-be51-17dd2e9d07cd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:37.567167 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ac36b5-1400-4a10-a94a-c1cafb4b5846 admin admin] PATH_INFO: `/v3/projects/4f392d7e74404c218bab9b5b44d17abc/users/90ed62fc171b47298023cb7601e08b59/roles/15f634d4a2d247ddab90c458e5e1c141` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:37.569134 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ac36b5-1400-4a10-a94a-c1cafb4b5846 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f392d7e74404c218bab9b5b44d17abc, user_id=90ed62fc171b47298023cb7601e08b59, role_id=15f634d4a2d247ddab90c458e5e1c141)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:37.569397 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30646782-b1f2-4c63-be51-17dd2e9d07cd admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:37.572891 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ac36b5-1400-4a10-a94a-c1cafb4b5846 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:37.589161 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1212/2432] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 24 10:03:37 2022] PUT /identity/v3/projects/4f392d7e74404c218bab9b5b44d17abc/users/90ed62fc171b47298023cb7601e08b59/roles/15f634d4a2d247ddab90c458e5e1c141 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:03:37.600246 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec877091-79de-4640-be3f-ddd61228de00 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:37.600536 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec877091-79de-4640-be3f-ddd61228de00 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:37.600837 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec877091-79de-4640-be3f-ddd61228de00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:37.638517 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-ec877091-79de-4640-be3f-ddd61228de00 None None] MFA Rules not processed for user `90ed62fc171b47298023cb7601e08b59`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:03:37.639854 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-ec877091-79de-4640-be3f-ddd61228de00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:37.661480 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1221/2433] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 24 10:03:37 2022] POST /identity/v3/users => generated 300 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 10:03:37.665563 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30646782-b1f2-4c63-be51-17dd2e9d07cd admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:37.676512 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30646782-b1f2-4c63-be51-17dd2e9d07cd admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:37.712250 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1213/2434] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 10:03:37 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:03:37.764401 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c617e9b4-b473-4d01-a937-f1d9cffeba79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:37.764401 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c617e9b4-b473-4d01-a937-f1d9cffeba79 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:37.764401 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c617e9b4-b473-4d01-a937-f1d9cffeba79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:37.764401 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c617e9b4-b473-4d01-a937-f1d9cffeba79 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:37.764910 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c617e9b4-b473-4d01-a937-f1d9cffeba79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:37.764910 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c617e9b4-b473-4d01-a937-f1d9cffeba79 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:37.764910 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1222/2435] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 24 10:03:37 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 10:03:37.765145 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec877091-79de-4640-be3f-ddd61228de00 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:37.765246 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c617e9b4-b473-4d01-a937-f1d9cffeba79 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:37.765639 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-c617e9b4-b473-4d01-a937-f1d9cffeba79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:37.769282 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec877091-79de-4640-be3f-ddd61228de00 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:37.776493 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88bb7b0-9e0e-4e12-8b02-9e70d517248d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:37.777171 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88bb7b0-9e0e-4e12-8b02-9e70d517248d admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:37.777459 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88bb7b0-9e0e-4e12-8b02-9e70d517248d admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:37.777732 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88bb7b0-9e0e-4e12-8b02-9e70d517248d admin admin] PATH_INFO: `/v3/projects/37fbb021aeb9463f9d85f1d04f3e1e4a/users/6832fdc882884cd8bc9071b44a70efbc/roles/15f634d4a2d247ddab90c458e5e1c141` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:37.786949 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:37.794424 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88bb7b0-9e0e-4e12-8b02-9e70d517248d admin admin] RBAC: Authorizing `identity:create_grant(project_id=37fbb021aeb9463f9d85f1d04f3e1e4a, user_id=6832fdc882884cd8bc9071b44a70efbc, role_id=15f634d4a2d247ddab90c458e5e1c141)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:37.797766 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88bb7b0-9e0e-4e12-8b02-9e70d517248d admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:37.825526 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c617e9b4-b473-4d01-a937-f1d9cffeba79 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:37.829458 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10314baf-0ed6-4681-a2ca-a017126cf732 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c32b83577c46488906fe409a790b24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:37.830068 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10314baf-0ed6-4681-a2ca-a017126cf732 service cinder] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:37.830350 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10314baf-0ed6-4681-a2ca-a017126cf732 service cinder] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:37.830622 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10314baf-0ed6-4681-a2ca-a017126cf732 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:37.839403 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10314baf-0ed6-4681-a2ca-a017126cf732 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:37.840916 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10314baf-0ed6-4681-a2ca-a017126cf732 service cinder] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:37.850194 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1223/2436] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 10:03:37 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 10:03:37.892391 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1214/2437] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 24 10:03:37 2022] PUT /identity/v3/projects/37fbb021aeb9463f9d85f1d04f3e1e4a/users/6832fdc882884cd8bc9071b44a70efbc/roles/15f634d4a2d247ddab90c458e5e1c141 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:03:37.894630 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10314baf-0ed6-4681-a2ca-a017126cf732 service cinder] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:37.906194 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10314baf-0ed6-4681-a2ca-a017126cf732 service cinder] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:37.921996 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88bb7b0-9e0e-4e12-8b02-9e70d517248d admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:37.935388 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:37.965619 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a207b4-e001-464a-9061-644d594c4dd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:37.966208 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a207b4-e001-464a-9061-644d594c4dd8 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:37.966486 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a207b4-e001-464a-9061-644d594c4dd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:37.966758 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a207b4-e001-464a-9061-644d594c4dd8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:37.967396 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a207b4-e001-464a-9061-644d594c4dd8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:37.969726 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a207b4-e001-464a-9061-644d594c4dd8 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:37.982291 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1224/2438] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 24 10:03:37 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:03:37.984984 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88bb7b0-9e0e-4e12-8b02-9e70d517248d admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:37.985110 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a207b4-e001-464a-9061-644d594c4dd8 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:37.985977 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d1545f-8b92-4f48-a432-3d7116ef33ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:37.986614 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d1545f-8b92-4f48-a432-3d7116ef33ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:37.986899 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d1545f-8b92-4f48-a432-3d7116ef33ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:37.987302 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d1545f-8b92-4f48-a432-3d7116ef33ef admin admin] PATH_INFO: `/v3/users/b3c74acd731547078091f85d1c769ea3` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:37.988480 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d1545f-8b92-4f48-a432-3d7116ef33ef admin admin] RBAC: Authorizing `identity:delete_user(user_id=b3c74acd731547078091f85d1c769ea3)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:37.990392 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d1545f-8b92-4f48-a432-3d7116ef33ef admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:37.993789 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a207b4-e001-464a-9061-644d594c4dd8 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:37.998502 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f98d83c-65d0-4f45-bcb5-6b69364b3cca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:37.999185 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f98d83c-65d0-4f45-bcb5-6b69364b3cca admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:37.999465 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f98d83c-65d0-4f45-bcb5-6b69364b3cca admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:37.999775 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f98d83c-65d0-4f45-bcb5-6b69364b3cca admin admin] PATH_INFO: `/v3/projects/37fbb021aeb9463f9d85f1d04f3e1e4a/users/6832fdc882884cd8bc9071b44a70efbc/roles/15f634d4a2d247ddab90c458e5e1c141` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:38.001667 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f98d83c-65d0-4f45-bcb5-6b69364b3cca admin admin] RBAC: Authorizing `identity:create_grant(project_id=37fbb021aeb9463f9d85f1d04f3e1e4a, user_id=6832fdc882884cd8bc9071b44a70efbc, role_id=15f634d4a2d247ddab90c458e5e1c141)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:38.004363 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f98d83c-65d0-4f45-bcb5-6b69364b3cca admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:38.014998 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1225/2439] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 24 10:03:37 2022] PUT /identity/v3/projects/37fbb021aeb9463f9d85f1d04f3e1e4a/users/6832fdc882884cd8bc9071b44a70efbc/roles/15f634d4a2d247ddab90c458e5e1c141 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:03:38.025368 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d67c4ed-e296-4265-a2c7-a19cbb01ef90 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:38.025368 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d67c4ed-e296-4265-a2c7-a19cbb01ef90 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:38.025541 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d67c4ed-e296-4265-a2c7-a19cbb01ef90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:38.057354 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-3d67c4ed-e296-4265-a2c7-a19cbb01ef90 None None] MFA Rules not processed for user `6832fdc882884cd8bc9071b44a70efbc`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:03:38.058562 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-3d67c4ed-e296-4265-a2c7-a19cbb01ef90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:38.126928 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1226/2440] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 10:03:38 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:03:38.137439 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d67c4ed-e296-4265-a2c7-a19cbb01ef90 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:38.138770 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-3d67c4ed-e296-4265-a2c7-a19cbb01ef90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:38.156082 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-e5d1545f-8b92-4f48-a432-3d7116ef33ef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b3c74acd731547078091f85d1c769ea3'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:38.163653 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-e5d1545f-8b92-4f48-a432-3d7116ef33ef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b3c74acd731547078091f85d1c769ea3'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:38.168746 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-e5d1545f-8b92-4f48-a432-3d7116ef33ef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b3c74acd731547078091f85d1c769ea3'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:38.176071 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-e5d1545f-8b92-4f48-a432-3d7116ef33ef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b3c74acd731547078091f85d1c769ea3'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:38.195641 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d67c4ed-e296-4265-a2c7-a19cbb01ef90 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:38.199225 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb11dfc-a621-4cdc-b9d8-4da14abc026d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c32b83577c46488906fe409a790b24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:38.199862 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb11dfc-a621-4cdc-b9d8-4da14abc026d service cinder] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:38.200147 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb11dfc-a621-4cdc-b9d8-4da14abc026d service cinder] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:38.200422 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb11dfc-a621-4cdc-b9d8-4da14abc026d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:38.209389 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb11dfc-a621-4cdc-b9d8-4da14abc026d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:38.210808 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb11dfc-a621-4cdc-b9d8-4da14abc026d service cinder] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:38.220638 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1227/2441] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 10:03:38 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 10:03:38.248606 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-e5d1545f-8b92-4f48-a432-3d7116ef33ef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b3c74acd731547078091f85d1c769ea3'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:38.256549 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1215/2442] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 24 10:03:37 2022] DELETE /identity/v3/users/b3c74acd731547078091f85d1c769ea3 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:03:38.272818 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f44557-efd3-4c6c-bae7-cdf36b9add83 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:38.272818 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f44557-efd3-4c6c-bae7-cdf36b9add83 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:38.272818 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f44557-efd3-4c6c-bae7-cdf36b9add83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:38.281939 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-64f44557-efd3-4c6c-bae7-cdf36b9add83 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:03:38.283145 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-64f44557-efd3-4c6c-bae7-cdf36b9add83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:38.319889 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:38.358361 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1228/2443] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 10:03:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:03:38.359496 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d1545f-8b92-4f48-a432-3d7116ef33ef admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:38.361085 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-e5d1545f-8b92-4f48-a432-3d7116ef33ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:38.417335 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d1545f-8b92-4f48-a432-3d7116ef33ef admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:38.421074 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeffc872-53eb-425d-9179-904dbe586e72 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e5260e798404e8bb12b25ca8ea97052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:38.421637 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeffc872-53eb-425d-9179-904dbe586e72 service neutron] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:38.421912 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeffc872-53eb-425d-9179-904dbe586e72 service neutron] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:38.422183 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeffc872-53eb-425d-9179-904dbe586e72 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:38.431342 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeffc872-53eb-425d-9179-904dbe586e72 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:38.432791 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeffc872-53eb-425d-9179-904dbe586e72 service neutron] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:38.442837 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1216/2444] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 10:03:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:03:38.829880 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f44557-efd3-4c6c-bae7-cdf36b9add83 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:38.831530 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-64f44557-efd3-4c6c-bae7-cdf36b9add83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:38.846954 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f44557-efd3-4c6c-bae7-cdf36b9add83 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:38.854522 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b81da9-035e-4734-a984-3ace2ed84f68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:38.854522 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b81da9-035e-4734-a984-3ace2ed84f68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:38.854522 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b81da9-035e-4734-a984-3ace2ed84f68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:38.854522 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b81da9-035e-4734-a984-3ace2ed84f68 admin admin] PATH_INFO: `/v3/projects/0efa2cdf9808421682838cfb83e31a38` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:38.855323 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b81da9-035e-4734-a984-3ace2ed84f68 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0efa2cdf9808421682838cfb83e31a38)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:38.857174 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b81da9-035e-4734-a984-3ace2ed84f68 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:38.981222 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-00b81da9-035e-4734-a984-3ace2ed84f68 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0efa2cdf9808421682838cfb83e31a38', 'request_id': 'req-00b81da9-035e-4734-a984-3ace2ed84f68'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:38.987447 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-00b81da9-035e-4734-a984-3ace2ed84f68 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0efa2cdf9808421682838cfb83e31a38', 'request_id': 'req-00b81da9-035e-4734-a984-3ace2ed84f68'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:38.992956 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-00b81da9-035e-4734-a984-3ace2ed84f68 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0efa2cdf9808421682838cfb83e31a38', 'request_id': 'req-00b81da9-035e-4734-a984-3ace2ed84f68'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:39.070819 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-00b81da9-035e-4734-a984-3ace2ed84f68 admin admin] The token cache is being invalidate because project 0efa2cdf9808421682838cfb83e31a38 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81602) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:03:39.071040 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-00b81da9-035e-4734-a984-3ace2ed84f68 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:39.071749 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-00b81da9-035e-4734-a984-3ace2ed84f68 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:39.073273 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1229/2445] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 24 10:03:38 2022] DELETE /identity/v3/projects/0efa2cdf9808421682838cfb83e31a38 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:03:39.077501 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeffc872-53eb-425d-9179-904dbe586e72 service neutron] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:39.087519 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeffc872-53eb-425d-9179-904dbe586e72 service neutron] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:39.092131 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:39.145200 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce79978-2090-4512-b494-4d22a54ba3ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:39.145840 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce79978-2090-4512-b494-4d22a54ba3ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:39.146122 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce79978-2090-4512-b494-4d22a54ba3ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:39.146398 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce79978-2090-4512-b494-4d22a54ba3ba admin admin] PATH_INFO: `/v3/users/90ed62fc171b47298023cb7601e08b59` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:39.147531 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce79978-2090-4512-b494-4d22a54ba3ba admin admin] RBAC: Authorizing `identity:delete_user(user_id=90ed62fc171b47298023cb7601e08b59)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:39.149478 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce79978-2090-4512-b494-4d22a54ba3ba admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:39.381842 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-cce79978-2090-4512-b494-4d22a54ba3ba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '90ed62fc171b47298023cb7601e08b59'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:39.381842 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-cce79978-2090-4512-b494-4d22a54ba3ba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '90ed62fc171b47298023cb7601e08b59'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:39.386196 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-cce79978-2090-4512-b494-4d22a54ba3ba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '90ed62fc171b47298023cb7601e08b59'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:39.393003 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-cce79978-2090-4512-b494-4d22a54ba3ba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '90ed62fc171b47298023cb7601e08b59'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:39.399494 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b81da9-035e-4734-a984-3ace2ed84f68 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:39.400984 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-00b81da9-035e-4734-a984-3ace2ed84f68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:39.454137 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-cce79978-2090-4512-b494-4d22a54ba3ba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '90ed62fc171b47298023cb7601e08b59'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:39.458779 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b81da9-035e-4734-a984-3ace2ed84f68 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:39.462649 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150718d1-35cb-4599-ad09-9c31e664bf7b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c32b83577c46488906fe409a790b24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:39.463099 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1217/2446] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 24 10:03:39 2022] DELETE /identity/v3/users/90ed62fc171b47298023cb7601e08b59 => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:03:39.464012 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150718d1-35cb-4599-ad09-9c31e664bf7b service cinder] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:39.464356 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150718d1-35cb-4599-ad09-9c31e664bf7b service cinder] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:39.464727 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150718d1-35cb-4599-ad09-9c31e664bf7b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:39.466777 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-150718d1-35cb-4599-ad09-9c31e664bf7b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:39.528182 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150718d1-35cb-4599-ad09-9c31e664bf7b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:39.529671 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150718d1-35cb-4599-ad09-9c31e664bf7b service cinder] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:39.539557 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1230/2447] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 10:03:39 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 10:03:39.919798 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce79978-2090-4512-b494-4d22a54ba3ba admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:39.921240 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-cce79978-2090-4512-b494-4d22a54ba3ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:39.987459 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce79978-2090-4512-b494-4d22a54ba3ba admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:39.992009 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e6ff94-1492-4269-ba91-f99f7bb8e933 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:39.992638 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e6ff94-1492-4269-ba91-f99f7bb8e933 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:39.992927 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e6ff94-1492-4269-ba91-f99f7bb8e933 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:39.993187 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e6ff94-1492-4269-ba91-f99f7bb8e933 admin admin] PATH_INFO: `/v3/projects/4f392d7e74404c218bab9b5b44d17abc` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:39.994311 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e6ff94-1492-4269-ba91-f99f7bb8e933 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4f392d7e74404c218bab9b5b44d17abc)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:39.996235 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e6ff94-1492-4269-ba91-f99f7bb8e933 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:40.106305 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:40.132711 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-89e6ff94-1492-4269-ba91-f99f7bb8e933 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4f392d7e74404c218bab9b5b44d17abc', 'request_id': 'req-89e6ff94-1492-4269-ba91-f99f7bb8e933'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:40.140721 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-89e6ff94-1492-4269-ba91-f99f7bb8e933 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4f392d7e74404c218bab9b5b44d17abc', 'request_id': 'req-89e6ff94-1492-4269-ba91-f99f7bb8e933'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:40.147930 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-89e6ff94-1492-4269-ba91-f99f7bb8e933 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4f392d7e74404c218bab9b5b44d17abc', 'request_id': 'req-89e6ff94-1492-4269-ba91-f99f7bb8e933'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:40.221274 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-89e6ff94-1492-4269-ba91-f99f7bb8e933 admin admin] The token cache is being invalidate because project 4f392d7e74404c218bab9b5b44d17abc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:03:40.221699 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-89e6ff94-1492-4269-ba91-f99f7bb8e933 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:40.222345 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-89e6ff94-1492-4269-ba91-f99f7bb8e933 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:40.223891 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1218/2448] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 24 10:03:39 2022] DELETE /identity/v3/projects/4f392d7e74404c218bab9b5b44d17abc => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:03:40.229636 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b15fe1-3e4f-45de-bf44-8b9da73741d4 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:40.230049 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b15fe1-3e4f-45de-bf44-8b9da73741d4 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:40.230216 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b15fe1-3e4f-45de-bf44-8b9da73741d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:40.250830 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-f8b15fe1-3e4f-45de-bf44-8b9da73741d4 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:03:40.254973 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-f8b15fe1-3e4f-45de-bf44-8b9da73741d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:40.265863 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:40.313005 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1231/2449] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 10:03:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:03:40.330661 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05f087b-6e9f-40e9-9947-cdbd2c14c132 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:40.330949 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05f087b-6e9f-40e9-9947-cdbd2c14c132 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:40.331233 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05f087b-6e9f-40e9-9947-cdbd2c14c132 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:40.351376 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-c05f087b-6e9f-40e9-9947-cdbd2c14c132 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:03:40.352620 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-c05f087b-6e9f-40e9-9947-cdbd2c14c132 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:40.361224 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1219/2450] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 10:03:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:03:40.365423 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b15fe1-3e4f-45de-bf44-8b9da73741d4 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:40.375299 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b15fe1-3e4f-45de-bf44-8b9da73741d4 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:40.380003 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e24cd1-99b6-44e3-be89-d29d1984d407 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:40.380612 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e24cd1-99b6-44e3-be89-d29d1984d407 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:40.380897 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e24cd1-99b6-44e3-be89-d29d1984d407 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:40.381180 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e24cd1-99b6-44e3-be89-d29d1984d407 admin admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:40.381824 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e24cd1-99b6-44e3-be89-d29d1984d407 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:03:40.382159 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e24cd1-99b6-44e3-be89-d29d1984d407 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:40.383776 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e24cd1-99b6-44e3-be89-d29d1984d407 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:40.395627 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1232/2451] 10.222.0.16 () {58 vars in 1153 bytes} [Wed Aug 24 10:03:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:03:40.400164 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05f087b-6e9f-40e9-9947-cdbd2c14c132 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:40.409969 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05f087b-6e9f-40e9-9947-cdbd2c14c132 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:40.414715 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da803da-d3f6-44f2-bc99-ee1599e1e976 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:40.415250 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da803da-d3f6-44f2-bc99-ee1599e1e976 admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:40.415528 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da803da-d3f6-44f2-bc99-ee1599e1e976 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:40.415832 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da803da-d3f6-44f2-bc99-ee1599e1e976 admin admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:40.416555 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da803da-d3f6-44f2-bc99-ee1599e1e976 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:40.418431 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da803da-d3f6-44f2-bc99-ee1599e1e976 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:40.525974 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1220/2452] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 24 10:03:40 2022] POST /identity/v3/projects => generated 367 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:03:40.530123 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e24cd1-99b6-44e3-be89-d29d1984d407 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:40.540576 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e24cd1-99b6-44e3-be89-d29d1984d407 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:40.596085 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302829fd-65cc-4b59-a035-800ecf9a7e45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:40.596683 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302829fd-65cc-4b59-a035-800ecf9a7e45 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:40.596961 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302829fd-65cc-4b59-a035-800ecf9a7e45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:40.597236 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302829fd-65cc-4b59-a035-800ecf9a7e45 admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:40.597846 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302829fd-65cc-4b59-a035-800ecf9a7e45 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:40.599769 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302829fd-65cc-4b59-a035-800ecf9a7e45 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:40.661374 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1233/2453] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 24 10:03:40 2022] POST /identity/v3/users => generated 284 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:03:40.665737 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da803da-d3f6-44f2-bc99-ee1599e1e976 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:40.676163 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da803da-d3f6-44f2-bc99-ee1599e1e976 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:40.681125 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee3ede7-58b4-444d-906e-2b07531ee327 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:40.681717 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee3ede7-58b4-444d-906e-2b07531ee327 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:40.681992 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee3ede7-58b4-444d-906e-2b07531ee327 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:40.682258 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee3ede7-58b4-444d-906e-2b07531ee327 admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:40.682930 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee3ede7-58b4-444d-906e-2b07531ee327 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:40.684552 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee3ede7-58b4-444d-906e-2b07531ee327 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:40.694445 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1221/2454] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 24 10:03:40 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:03:40.698218 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302829fd-65cc-4b59-a035-800ecf9a7e45 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:40.708970 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302829fd-65cc-4b59-a035-800ecf9a7e45 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:40.713703 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9041e7-b00a-4688-837a-76416c11c625 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:40.714288 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9041e7-b00a-4688-837a-76416c11c625 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:40.714568 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9041e7-b00a-4688-837a-76416c11c625 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:40.714840 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9041e7-b00a-4688-837a-76416c11c625 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:40.715473 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9041e7-b00a-4688-837a-76416c11c625 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:40.717119 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9041e7-b00a-4688-837a-76416c11c625 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:40.728027 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1234/2455] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 24 10:03:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:03:40.731944 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee3ede7-58b4-444d-906e-2b07531ee327 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:40.741747 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee3ede7-58b4-444d-906e-2b07531ee327 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:40.746403 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794d6ef1-352a-4f0f-8ba8-744a45827efc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:40.747048 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794d6ef1-352a-4f0f-8ba8-744a45827efc admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:40.747324 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794d6ef1-352a-4f0f-8ba8-744a45827efc admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:40.747619 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794d6ef1-352a-4f0f-8ba8-744a45827efc admin admin] PATH_INFO: `/v3/projects/4556da2c51c84e2d85c823d4041d0171/users/dcf743aadf0d47d594eb25c787302b27/roles/136cdf78e00a426997dd4e37faff489b` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:40.761885 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794d6ef1-352a-4f0f-8ba8-744a45827efc admin admin] RBAC: Authorizing `identity:create_grant(project_id=4556da2c51c84e2d85c823d4041d0171, user_id=dcf743aadf0d47d594eb25c787302b27, role_id=136cdf78e00a426997dd4e37faff489b)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:40.765171 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794d6ef1-352a-4f0f-8ba8-744a45827efc admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:40.871522 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1222/2456] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 24 10:03:40 2022] PUT /identity/v3/projects/4556da2c51c84e2d85c823d4041d0171/users/dcf743aadf0d47d594eb25c787302b27/roles/136cdf78e00a426997dd4e37faff489b => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:03:40.881618 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144d1dfd-aff5-4a22-baca-30c886a4dcba None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:40.881901 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144d1dfd-aff5-4a22-baca-30c886a4dcba None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:40.886999 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144d1dfd-aff5-4a22-baca-30c886a4dcba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:40.911254 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-144d1dfd-aff5-4a22-baca-30c886a4dcba None None] MFA Rules not processed for user `dcf743aadf0d47d594eb25c787302b27`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:03:40.912589 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-144d1dfd-aff5-4a22-baca-30c886a4dcba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:40.974996 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1235/2457] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 10:03:40 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:03:40.979519 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794d6ef1-352a-4f0f-8ba8-744a45827efc admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:41.040349 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794d6ef1-352a-4f0f-8ba8-744a45827efc admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:41.044993 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51ac9e7-8529-44a7-85f8-df97b696f953 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:41.045600 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51ac9e7-8529-44a7-85f8-df97b696f953 admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:41.045883 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51ac9e7-8529-44a7-85f8-df97b696f953 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:41.046158 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51ac9e7-8529-44a7-85f8-df97b696f953 admin admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:41.046776 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51ac9e7-8529-44a7-85f8-df97b696f953 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:41.048743 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51ac9e7-8529-44a7-85f8-df97b696f953 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:41.141480 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1223/2458] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 24 10:03:40 2022] POST /identity/v3/projects => generated 367 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:03:41.145734 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144d1dfd-aff5-4a22-baca-30c886a4dcba None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:41.156407 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144d1dfd-aff5-4a22-baca-30c886a4dcba None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:41.165168 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:41.217975 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f4be94-c4d1-4889-a29e-dfb350d22af7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:41.218612 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f4be94-c4d1-4889-a29e-dfb350d22af7 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:41.218893 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f4be94-c4d1-4889-a29e-dfb350d22af7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:41.219164 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f4be94-c4d1-4889-a29e-dfb350d22af7 admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:41.219845 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f4be94-c4d1-4889-a29e-dfb350d22af7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:41.221756 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f4be94-c4d1-4889-a29e-dfb350d22af7 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:41.324430 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1236/2459] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 24 10:03:41 2022] POST /identity/v3/users => generated 290 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:03:41.335554 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51ac9e7-8529-44a7-85f8-df97b696f953 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:41.341033 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51ac9e7-8529-44a7-85f8-df97b696f953 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:41.346670 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6387da1-7889-4967-9516-da10dbb216a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:41.347277 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6387da1-7889-4967-9516-da10dbb216a0 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:41.347556 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6387da1-7889-4967-9516-da10dbb216a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:41.347857 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6387da1-7889-4967-9516-da10dbb216a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:41.348499 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6387da1-7889-4967-9516-da10dbb216a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:41.350090 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6387da1-7889-4967-9516-da10dbb216a0 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:41.360974 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1224/2460] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 24 10:03:41 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 10:03:41.365146 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f4be94-c4d1-4889-a29e-dfb350d22af7 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:41.375120 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f4be94-c4d1-4889-a29e-dfb350d22af7 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:41.381293 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616f3b0d-2119-491e-955b-ef2a5615bedb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:41.382050 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616f3b0d-2119-491e-955b-ef2a5615bedb admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:41.382945 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616f3b0d-2119-491e-955b-ef2a5615bedb admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:41.388203 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616f3b0d-2119-491e-955b-ef2a5615bedb admin admin] PATH_INFO: `/v3/projects/a5d90d6bc57e452aa5b840578e7becf0/users/8f8b34867b9a4499b5c6aa77885eb6bd/roles/15f634d4a2d247ddab90c458e5e1c141` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:41.401279 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616f3b0d-2119-491e-955b-ef2a5615bedb admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5d90d6bc57e452aa5b840578e7becf0, user_id=8f8b34867b9a4499b5c6aa77885eb6bd, role_id=15f634d4a2d247ddab90c458e5e1c141)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:41.404059 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616f3b0d-2119-491e-955b-ef2a5615bedb admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:41.483754 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1237/2461] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 24 10:03:41 2022] PUT /identity/v3/projects/a5d90d6bc57e452aa5b840578e7becf0/users/8f8b34867b9a4499b5c6aa77885eb6bd/roles/15f634d4a2d247ddab90c458e5e1c141 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:03:41.487626 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6387da1-7889-4967-9516-da10dbb216a0 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:41.497567 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6387da1-7889-4967-9516-da10dbb216a0 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:41.524088 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:41.555160 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3995ae89-e49f-43c5-9941-f49c24f5007e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:41.555906 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3995ae89-e49f-43c5-9941-f49c24f5007e admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:41.556221 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3995ae89-e49f-43c5-9941-f49c24f5007e admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:41.556499 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3995ae89-e49f-43c5-9941-f49c24f5007e admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:41.557170 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3995ae89-e49f-43c5-9941-f49c24f5007e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:41.558810 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3995ae89-e49f-43c5-9941-f49c24f5007e admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:41.568571 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1225/2462] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 24 10:03:41 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:03:41.572617 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616f3b0d-2119-491e-955b-ef2a5615bedb admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:41.583547 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616f3b0d-2119-491e-955b-ef2a5615bedb admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:41.588310 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fca46c5-9d49-414e-aa19-5944d7cedaf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:41.588986 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fca46c5-9d49-414e-aa19-5944d7cedaf2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:41.589262 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fca46c5-9d49-414e-aa19-5944d7cedaf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:41.589530 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fca46c5-9d49-414e-aa19-5944d7cedaf2 admin admin] PATH_INFO: `/v3/projects/a5d90d6bc57e452aa5b840578e7becf0/users/8f8b34867b9a4499b5c6aa77885eb6bd/roles/15f634d4a2d247ddab90c458e5e1c141` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:41.591429 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fca46c5-9d49-414e-aa19-5944d7cedaf2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5d90d6bc57e452aa5b840578e7becf0, user_id=8f8b34867b9a4499b5c6aa77885eb6bd, role_id=15f634d4a2d247ddab90c458e5e1c141)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:41.594683 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fca46c5-9d49-414e-aa19-5944d7cedaf2 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:41.604678 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1238/2463] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 24 10:03:41 2022] PUT /identity/v3/projects/a5d90d6bc57e452aa5b840578e7becf0/users/8f8b34867b9a4499b5c6aa77885eb6bd/roles/15f634d4a2d247ddab90c458e5e1c141 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:03:41.616052 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19013b4d-eaf7-498d-a257-4869500e96cc None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:41.616341 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19013b4d-eaf7-498d-a257-4869500e96cc None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:41.616617 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19013b4d-eaf7-498d-a257-4869500e96cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:41.645199 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-19013b4d-eaf7-498d-a257-4869500e96cc None None] MFA Rules not processed for user `8f8b34867b9a4499b5c6aa77885eb6bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:03:41.646448 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-19013b4d-eaf7-498d-a257-4869500e96cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:41.714227 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1226/2464] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 10:03:41 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:03:41.725337 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fca46c5-9d49-414e-aa19-5944d7cedaf2 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:41.726987 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-4fca46c5-9d49-414e-aa19-5944d7cedaf2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:41.783759 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fca46c5-9d49-414e-aa19-5944d7cedaf2 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:41.787315 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf1002c-5e1d-408c-8f57-98e8b9daffb2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c32b83577c46488906fe409a790b24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:41.787946 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf1002c-5e1d-408c-8f57-98e8b9daffb2 service cinder] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:41.788220 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf1002c-5e1d-408c-8f57-98e8b9daffb2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:41.788490 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf1002c-5e1d-408c-8f57-98e8b9daffb2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:41.797270 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf1002c-5e1d-408c-8f57-98e8b9daffb2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:41.798668 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf1002c-5e1d-408c-8f57-98e8b9daffb2 service cinder] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:41.808088 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1239/2465] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 10:03:41 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:03:42.064652 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19013b4d-eaf7-498d-a257-4869500e96cc None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:42.074478 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19013b4d-eaf7-498d-a257-4869500e96cc None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:42.078073 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dcdd43-1388-43d5-add4-17a27b2faefe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c32b83577c46488906fe409a790b24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:42.078669 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dcdd43-1388-43d5-add4-17a27b2faefe service cinder] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:42.078943 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dcdd43-1388-43d5-add4-17a27b2faefe service cinder] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:42.079472 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dcdd43-1388-43d5-add4-17a27b2faefe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:42.134964 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57dcdd43-1388-43d5-add4-17a27b2faefe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:42.136400 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57dcdd43-1388-43d5-add4-17a27b2faefe service cinder] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:42.145020 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1227/2466] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 10:03:42 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:03:46.089279 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf1002c-5e1d-408c-8f57-98e8b9daffb2 service cinder] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:46.090790 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-5cf1002c-5e1d-408c-8f57-98e8b9daffb2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:46.169436 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf1002c-5e1d-408c-8f57-98e8b9daffb2 service cinder] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:46.175148 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435840b7-536a-43b0-8546-e3d157648407 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:46.175832 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435840b7-536a-43b0-8546-e3d157648407 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:46.176092 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435840b7-536a-43b0-8546-e3d157648407 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:46.176366 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435840b7-536a-43b0-8546-e3d157648407 admin admin] PATH_INFO: `/v3/users/566eca13aaff4761b66e5c5a16818651` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:46.177561 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435840b7-536a-43b0-8546-e3d157648407 admin admin] RBAC: Authorizing `identity:delete_user(user_id=566eca13aaff4761b66e5c5a16818651)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:46.179465 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435840b7-536a-43b0-8546-e3d157648407 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:46.190185 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:46.399474 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-435840b7-536a-43b0-8546-e3d157648407 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '566eca13aaff4761b66e5c5a16818651'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:46.400157 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-435840b7-536a-43b0-8546-e3d157648407 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '566eca13aaff4761b66e5c5a16818651'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:46.478054 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-435840b7-536a-43b0-8546-e3d157648407 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '566eca13aaff4761b66e5c5a16818651'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:46.478668 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-435840b7-536a-43b0-8546-e3d157648407 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '566eca13aaff4761b66e5c5a16818651'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:46.486098 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-435840b7-536a-43b0-8546-e3d157648407 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '566eca13aaff4761b66e5c5a16818651'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:46.502949 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1240/2467] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 24 10:03:46 2022] DELETE /identity/v3/users/566eca13aaff4761b66e5c5a16818651 => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:03:46.507406 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62064ef1-065c-40f2-837c-15dd1f16714f None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:46.507947 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62064ef1-065c-40f2-837c-15dd1f16714f None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:46.508041 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62064ef1-065c-40f2-837c-15dd1f16714f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:46.536116 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-62064ef1-065c-40f2-837c-15dd1f16714f None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:03:46.537348 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-62064ef1-065c-40f2-837c-15dd1f16714f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:46.577892 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:46.599749 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1228/2468] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 10:03:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:03:46.609505 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435840b7-536a-43b0-8546-e3d157648407 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:46.611072 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-435840b7-536a-43b0-8546-e3d157648407 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:46.671295 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435840b7-536a-43b0-8546-e3d157648407 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:46.675127 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b096c72b-1c02-438e-ab82-544a12627203 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e5260e798404e8bb12b25ca8ea97052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:46.675772 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b096c72b-1c02-438e-ab82-544a12627203 service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:46.676055 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b096c72b-1c02-438e-ab82-544a12627203 service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:46.676450 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b096c72b-1c02-438e-ab82-544a12627203 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:46.685425 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b096c72b-1c02-438e-ab82-544a12627203 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:46.686865 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b096c72b-1c02-438e-ab82-544a12627203 service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:46.697076 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1241/2469] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 10:03:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:03:47.171639 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62064ef1-065c-40f2-837c-15dd1f16714f None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:47.173131 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-62064ef1-065c-40f2-837c-15dd1f16714f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:47.185989 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62064ef1-065c-40f2-837c-15dd1f16714f None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:47.191763 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b48818-db29-48c4-9f8a-94bfcdfb6866 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:47.192926 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b48818-db29-48c4-9f8a-94bfcdfb6866 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:47.193275 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b48818-db29-48c4-9f8a-94bfcdfb6866 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:47.193618 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b48818-db29-48c4-9f8a-94bfcdfb6866 admin admin] PATH_INFO: `/v3/projects/5e04d6f93aff44b39f53db24f894e8bf` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:47.195229 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b48818-db29-48c4-9f8a-94bfcdfb6866 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5e04d6f93aff44b39f53db24f894e8bf)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:47.197392 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b48818-db29-48c4-9f8a-94bfcdfb6866 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:47.365333 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-f8b48818-db29-48c4-9f8a-94bfcdfb6866 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5e04d6f93aff44b39f53db24f894e8bf', 'request_id': 'req-f8b48818-db29-48c4-9f8a-94bfcdfb6866'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:47.372063 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-f8b48818-db29-48c4-9f8a-94bfcdfb6866 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5e04d6f93aff44b39f53db24f894e8bf', 'request_id': 'req-f8b48818-db29-48c4-9f8a-94bfcdfb6866'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:47.378403 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-f8b48818-db29-48c4-9f8a-94bfcdfb6866 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5e04d6f93aff44b39f53db24f894e8bf', 'request_id': 'req-f8b48818-db29-48c4-9f8a-94bfcdfb6866'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:47.510432 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-f8b48818-db29-48c4-9f8a-94bfcdfb6866 admin admin] The token cache is being invalidate because project 5e04d6f93aff44b39f53db24f894e8bf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:03:47.511124 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-f8b48818-db29-48c4-9f8a-94bfcdfb6866 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:47.511351 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-f8b48818-db29-48c4-9f8a-94bfcdfb6866 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:47.512927 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1229/2470] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 24 10:03:47 2022] DELETE /identity/v3/projects/5e04d6f93aff44b39f53db24f894e8bf => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:03:47.518264 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb67a97-565f-489e-8979-c71c40f5658e None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:47.518713 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb67a97-565f-489e-8979-c71c40f5658e None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:47.518824 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb67a97-565f-489e-8979-c71c40f5658e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:47.567302 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-0bb67a97-565f-489e-8979-c71c40f5658e None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:03:47.568630 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-0bb67a97-565f-489e-8979-c71c40f5658e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:47.630876 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1242/2471] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 10:03:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:03:47.638225 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e424287-ffba-4cab-9f33-04d3bba180db None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:47.638516 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e424287-ffba-4cab-9f33-04d3bba180db None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:47.638794 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e424287-ffba-4cab-9f33-04d3bba180db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:47.660309 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-5e424287-ffba-4cab-9f33-04d3bba180db None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:03:47.661612 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-5e424287-ffba-4cab-9f33-04d3bba180db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:47.669895 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1230/2472] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 10:03:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:03:47.675001 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb67a97-565f-489e-8979-c71c40f5658e None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:47.685835 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb67a97-565f-489e-8979-c71c40f5658e None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:47.690461 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50f7e0f-a967-4f79-981e-e5c1a51d1e91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:47.691061 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50f7e0f-a967-4f79-981e-e5c1a51d1e91 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:47.691342 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50f7e0f-a967-4f79-981e-e5c1a51d1e91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:47.691632 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50f7e0f-a967-4f79-981e-e5c1a51d1e91 admin admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:47.692280 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50f7e0f-a967-4f79-981e-e5c1a51d1e91 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:03:47.692650 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50f7e0f-a967-4f79-981e-e5c1a51d1e91 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:47.694225 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50f7e0f-a967-4f79-981e-e5c1a51d1e91 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:47.695766 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:47.707263 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1243/2473] 10.222.0.16 () {58 vars in 1153 bytes} [Wed Aug 24 10:03:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:03:47.711888 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e424287-ffba-4cab-9f33-04d3bba180db None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:47.721853 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e424287-ffba-4cab-9f33-04d3bba180db None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:47.726850 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527d72fe-0c5e-4640-b98f-9a8db438e306 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:47.727454 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527d72fe-0c5e-4640-b98f-9a8db438e306 admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:47.727766 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527d72fe-0c5e-4640-b98f-9a8db438e306 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:47.728043 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527d72fe-0c5e-4640-b98f-9a8db438e306 admin admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:47.728666 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527d72fe-0c5e-4640-b98f-9a8db438e306 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:47.730589 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527d72fe-0c5e-4640-b98f-9a8db438e306 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:47.854902 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1231/2474] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 24 10:03:47 2022] POST /identity/v3/projects => generated 405 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:03:47.856998 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50f7e0f-a967-4f79-981e-e5c1a51d1e91 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:47.868335 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50f7e0f-a967-4f79-981e-e5c1a51d1e91 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:47.942155 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b1e072-7090-4f51-b4db-57e833ad0969 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:47.942832 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b1e072-7090-4f51-b4db-57e833ad0969 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:47.943244 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b1e072-7090-4f51-b4db-57e833ad0969 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:47.943516 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b1e072-7090-4f51-b4db-57e833ad0969 admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:47.944260 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b1e072-7090-4f51-b4db-57e833ad0969 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:47.946260 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b1e072-7090-4f51-b4db-57e833ad0969 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:48.094902 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1244/2475] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 24 10:03:47 2022] POST /identity/v3/users => generated 303 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:03:48.099466 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527d72fe-0c5e-4640-b98f-9a8db438e306 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:48.109726 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527d72fe-0c5e-4640-b98f-9a8db438e306 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:48.114832 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6449a2f5-0b6b-4d5f-abe4-3a574f2c3c7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:48.115464 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6449a2f5-0b6b-4d5f-abe4-3a574f2c3c7c admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:48.115843 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6449a2f5-0b6b-4d5f-abe4-3a574f2c3c7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:48.116164 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6449a2f5-0b6b-4d5f-abe4-3a574f2c3c7c admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:48.116815 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6449a2f5-0b6b-4d5f-abe4-3a574f2c3c7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:48.119025 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6449a2f5-0b6b-4d5f-abe4-3a574f2c3c7c admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:48.129971 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1232/2476] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 24 10:03:48 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:03:48.136803 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b1e072-7090-4f51-b4db-57e833ad0969 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:48.152725 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b1e072-7090-4f51-b4db-57e833ad0969 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:48.157598 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c81aea4-c505-402c-81e2-acca94949039 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:48.159116 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c81aea4-c505-402c-81e2-acca94949039 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:48.159116 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c81aea4-c505-402c-81e2-acca94949039 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:48.161041 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c81aea4-c505-402c-81e2-acca94949039 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:48.161041 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c81aea4-c505-402c-81e2-acca94949039 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:48.168496 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c81aea4-c505-402c-81e2-acca94949039 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:48.178695 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1245/2477] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 24 10:03:48 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:03:48.182799 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6449a2f5-0b6b-4d5f-abe4-3a574f2c3c7c admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:48.192727 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6449a2f5-0b6b-4d5f-abe4-3a574f2c3c7c admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:48.197393 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d1af07-4d3f-49ca-a6d3-63ccb5039e66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:48.198079 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d1af07-4d3f-49ca-a6d3-63ccb5039e66 admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:48.198343 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d1af07-4d3f-49ca-a6d3-63ccb5039e66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:48.198619 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d1af07-4d3f-49ca-a6d3-63ccb5039e66 admin admin] PATH_INFO: `/v3/projects/9b3557c491154848b44751ecdbc750b0/users/006b0ef150bd4d788f811b39f9f25100/roles/136cdf78e00a426997dd4e37faff489b` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:48.215066 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d1af07-4d3f-49ca-a6d3-63ccb5039e66 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b3557c491154848b44751ecdbc750b0, user_id=006b0ef150bd4d788f811b39f9f25100, role_id=136cdf78e00a426997dd4e37faff489b)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:48.219300 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d1af07-4d3f-49ca-a6d3-63ccb5039e66 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:48.298302 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1233/2478] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 24 10:03:48 2022] PUT /identity/v3/projects/9b3557c491154848b44751ecdbc750b0/users/006b0ef150bd4d788f811b39f9f25100/roles/136cdf78e00a426997dd4e37faff489b => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:03:48.371142 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85242e92-aafc-4880-aad7-8775d083678f None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:48.371518 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85242e92-aafc-4880-aad7-8775d083678f None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:48.371912 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85242e92-aafc-4880-aad7-8775d083678f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:48.400572 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-85242e92-aafc-4880-aad7-8775d083678f None None] MFA Rules not processed for user `006b0ef150bd4d788f811b39f9f25100`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:03:48.401798 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-85242e92-aafc-4880-aad7-8775d083678f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:48.467210 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1246/2479] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 10:03:48 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:03:48.472239 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d1af07-4d3f-49ca-a6d3-63ccb5039e66 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:48.538860 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:48.549838 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d1af07-4d3f-49ca-a6d3-63ccb5039e66 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:48.554676 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12fc335-794d-4988-b0cf-66d3709de49d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:48.555289 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12fc335-794d-4988-b0cf-66d3709de49d admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:48.555575 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12fc335-794d-4988-b0cf-66d3709de49d admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:48.555876 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12fc335-794d-4988-b0cf-66d3709de49d admin admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:48.556808 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12fc335-794d-4988-b0cf-66d3709de49d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:48.558409 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12fc335-794d-4988-b0cf-66d3709de49d admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:48.652216 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1234/2480] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 24 10:03:48 2022] POST /identity/v3/projects => generated 403 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:03:48.658301 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85242e92-aafc-4880-aad7-8775d083678f None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:48.670420 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85242e92-aafc-4880-aad7-8775d083678f None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:48.726576 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e590b09-8886-4e48-aa4f-ce7c8c235d7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:48.727185 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e590b09-8886-4e48-aa4f-ce7c8c235d7d admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:48.727465 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e590b09-8886-4e48-aa4f-ce7c8c235d7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:48.727781 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e590b09-8886-4e48-aa4f-ce7c8c235d7d admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:48.728417 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e590b09-8886-4e48-aa4f-ce7c8c235d7d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:48.730328 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e590b09-8886-4e48-aa4f-ce7c8c235d7d admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:48.739481 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:48.856439 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1247/2481] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 24 10:03:48 2022] POST /identity/v3/users => generated 308 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 10:03:48.862857 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12fc335-794d-4988-b0cf-66d3709de49d admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:48.874422 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12fc335-794d-4988-b0cf-66d3709de49d admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:48.879536 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f52a3f8-bc2f-48d5-92dd-ebca0b1f84a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:48.880414 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f52a3f8-bc2f-48d5-92dd-ebca0b1f84a6 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:48.880889 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f52a3f8-bc2f-48d5-92dd-ebca0b1f84a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:48.883869 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f52a3f8-bc2f-48d5-92dd-ebca0b1f84a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:48.884756 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f52a3f8-bc2f-48d5-92dd-ebca0b1f84a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:48.886379 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f52a3f8-bc2f-48d5-92dd-ebca0b1f84a6 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:48.902667 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1235/2482] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 24 10:03:48 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:03:48.903760 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e590b09-8886-4e48-aa4f-ce7c8c235d7d admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:48.920900 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e590b09-8886-4e48-aa4f-ce7c8c235d7d admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:48.931631 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9372e2b9-63d3-45b9-b7e3-49259b6fdad4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:48.931631 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9372e2b9-63d3-45b9-b7e3-49259b6fdad4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:48.931631 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9372e2b9-63d3-45b9-b7e3-49259b6fdad4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:48.931631 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9372e2b9-63d3-45b9-b7e3-49259b6fdad4 admin admin] PATH_INFO: `/v3/projects/71fd88e02e834acabd1f8c9e660ff869/users/79c38cb764724d1ca799d0593d0d7b1f/roles/15f634d4a2d247ddab90c458e5e1c141` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:48.945502 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9372e2b9-63d3-45b9-b7e3-49259b6fdad4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=71fd88e02e834acabd1f8c9e660ff869, user_id=79c38cb764724d1ca799d0593d0d7b1f, role_id=15f634d4a2d247ddab90c458e5e1c141)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:48.948269 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9372e2b9-63d3-45b9-b7e3-49259b6fdad4 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:49.062041 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1248/2483] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 24 10:03:48 2022] PUT /identity/v3/projects/71fd88e02e834acabd1f8c9e660ff869/users/79c38cb764724d1ca799d0593d0d7b1f/roles/15f634d4a2d247ddab90c458e5e1c141 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 10:03:49.066799 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f52a3f8-bc2f-48d5-92dd-ebca0b1f84a6 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:49.076961 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f52a3f8-bc2f-48d5-92dd-ebca0b1f84a6 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:49.133161 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3f1876-ae57-4f0f-a97e-5d3667e39fc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:49.133769 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3f1876-ae57-4f0f-a97e-5d3667e39fc1 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:49.134084 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3f1876-ae57-4f0f-a97e-5d3667e39fc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:49.134325 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3f1876-ae57-4f0f-a97e-5d3667e39fc1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:49.134965 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3f1876-ae57-4f0f-a97e-5d3667e39fc1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:49.136610 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3f1876-ae57-4f0f-a97e-5d3667e39fc1 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:49.147174 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1236/2484] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 24 10:03:49 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:03:49.151693 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9372e2b9-63d3-45b9-b7e3-49259b6fdad4 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:49.162384 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9372e2b9-63d3-45b9-b7e3-49259b6fdad4 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:49.167678 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9b5cc3-9ac7-4597-a955-78b89729874d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:49.168407 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9b5cc3-9ac7-4597-a955-78b89729874d admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:49.168687 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9b5cc3-9ac7-4597-a955-78b89729874d admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:49.168957 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9b5cc3-9ac7-4597-a955-78b89729874d admin admin] PATH_INFO: `/v3/projects/71fd88e02e834acabd1f8c9e660ff869/users/79c38cb764724d1ca799d0593d0d7b1f/roles/15f634d4a2d247ddab90c458e5e1c141` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:49.171163 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9b5cc3-9ac7-4597-a955-78b89729874d admin admin] RBAC: Authorizing `identity:create_grant(project_id=71fd88e02e834acabd1f8c9e660ff869, user_id=79c38cb764724d1ca799d0593d0d7b1f, role_id=15f634d4a2d247ddab90c458e5e1c141)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:49.173833 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9b5cc3-9ac7-4597-a955-78b89729874d admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:49.195864 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1249/2485] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 24 10:03:49 2022] PUT /identity/v3/projects/71fd88e02e834acabd1f8c9e660ff869/users/79c38cb764724d1ca799d0593d0d7b1f/roles/15f634d4a2d247ddab90c458e5e1c141 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 10:03:49.204873 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72e5ee3-9b5a-44f9-a04d-0f8415d0d6ca None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:49.205276 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72e5ee3-9b5a-44f9-a04d-0f8415d0d6ca None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:49.206113 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72e5ee3-9b5a-44f9-a04d-0f8415d0d6ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:49.235240 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-e72e5ee3-9b5a-44f9-a04d-0f8415d0d6ca None None] MFA Rules not processed for user `79c38cb764724d1ca799d0593d0d7b1f`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:03:49.238909 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-e72e5ee3-9b5a-44f9-a04d-0f8415d0d6ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:49.306152 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1237/2486] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 10:03:49 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:03:49.317312 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9b5cc3-9ac7-4597-a955-78b89729874d admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:49.319112 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-2c9b5cc3-9ac7-4597-a955-78b89729874d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:49.381664 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9b5cc3-9ac7-4597-a955-78b89729874d admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:49.385386 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa1eea2-6ae9-4181-b028-50dbfdf8d874 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c32b83577c46488906fe409a790b24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:49.385997 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa1eea2-6ae9-4181-b028-50dbfdf8d874 service cinder] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:49.386283 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa1eea2-6ae9-4181-b028-50dbfdf8d874 service cinder] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:49.386560 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa1eea2-6ae9-4181-b028-50dbfdf8d874 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:49.395443 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa1eea2-6ae9-4181-b028-50dbfdf8d874 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:49.396910 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa1eea2-6ae9-4181-b028-50dbfdf8d874 service cinder] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:49.406403 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1250/2487] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 10:03:49 2022] GET /identity/v3/auth/tokens => generated 3347 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:03:49.586213 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72e5ee3-9b5a-44f9-a04d-0f8415d0d6ca None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:49.677944 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:49.692882 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72e5ee3-9b5a-44f9-a04d-0f8415d0d6ca None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:49.697568 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff05f91-6f19-465d-b16f-93ef99a59247 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:49.698168 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff05f91-6f19-465d-b16f-93ef99a59247 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:49.698451 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff05f91-6f19-465d-b16f-93ef99a59247 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:49.698726 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff05f91-6f19-465d-b16f-93ef99a59247 admin admin] PATH_INFO: `/v3/users/006b0ef150bd4d788f811b39f9f25100` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:49.699886 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff05f91-6f19-465d-b16f-93ef99a59247 admin admin] RBAC: Authorizing `identity:delete_user(user_id=006b0ef150bd4d788f811b39f9f25100)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:49.701917 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff05f91-6f19-465d-b16f-93ef99a59247 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:49.935462 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-1ff05f91-6f19-465d-b16f-93ef99a59247 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '006b0ef150bd4d788f811b39f9f25100'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:49.936429 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-1ff05f91-6f19-465d-b16f-93ef99a59247 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '006b0ef150bd4d788f811b39f9f25100'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:49.960503 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-1ff05f91-6f19-465d-b16f-93ef99a59247 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '006b0ef150bd4d788f811b39f9f25100'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:49.968828 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-1ff05f91-6f19-465d-b16f-93ef99a59247 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '006b0ef150bd4d788f811b39f9f25100'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:50.066018 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-1ff05f91-6f19-465d-b16f-93ef99a59247 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '006b0ef150bd4d788f811b39f9f25100'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:50.075970 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1238/2488] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 24 10:03:49 2022] DELETE /identity/v3/users/006b0ef150bd4d788f811b39f9f25100 => generated 0 bytes in 490 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:03:50.081487 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cced94-1fb2-4d81-b132-075d835d3223 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:50.082392 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cced94-1fb2-4d81-b132-075d835d3223 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:50.082833 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cced94-1fb2-4d81-b132-075d835d3223 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:50.103920 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-92cced94-1fb2-4d81-b132-075d835d3223 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:03:50.105215 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-92cced94-1fb2-4d81-b132-075d835d3223 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:50.169640 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1251/2489] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 10:03:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:03:50.180363 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff05f91-6f19-465d-b16f-93ef99a59247 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:50.181919 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-1ff05f91-6f19-465d-b16f-93ef99a59247 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:50.248923 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff05f91-6f19-465d-b16f-93ef99a59247 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:50.253238 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e502f5f7-d7ec-4dfd-881e-c6c267d03805 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e5260e798404e8bb12b25ca8ea97052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:50.253844 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e502f5f7-d7ec-4dfd-881e-c6c267d03805 service neutron] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:50.254122 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e502f5f7-d7ec-4dfd-881e-c6c267d03805 service neutron] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:50.254400 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e502f5f7-d7ec-4dfd-881e-c6c267d03805 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:50.264143 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e502f5f7-d7ec-4dfd-881e-c6c267d03805 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:50.265769 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e502f5f7-d7ec-4dfd-881e-c6c267d03805 service neutron] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:50.275838 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1239/2490] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 10:03:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:03:50.690917 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cced94-1fb2-4d81-b132-075d835d3223 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:50.692398 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-92cced94-1fb2-4d81-b132-075d835d3223 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:50.712846 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cced94-1fb2-4d81-b132-075d835d3223 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:50.718992 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da85562-4729-4b2e-a517-0f7b31b98fab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:50.719669 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da85562-4729-4b2e-a517-0f7b31b98fab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:50.719960 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da85562-4729-4b2e-a517-0f7b31b98fab admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:50.720229 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da85562-4729-4b2e-a517-0f7b31b98fab admin admin] PATH_INFO: `/v3/projects/9b3557c491154848b44751ecdbc750b0` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:50.721984 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da85562-4729-4b2e-a517-0f7b31b98fab admin admin] RBAC: Authorizing `identity:delete_project(project_id=9b3557c491154848b44751ecdbc750b0)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:50.723928 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da85562-4729-4b2e-a517-0f7b31b98fab admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:50.888637 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-4da85562-4729-4b2e-a517-0f7b31b98fab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9b3557c491154848b44751ecdbc750b0', 'request_id': 'req-4da85562-4729-4b2e-a517-0f7b31b98fab'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:50.888637 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:INSERT=4,keystone:SELECT=92,keystone:DELETE=4 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:50.893875 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-4da85562-4729-4b2e-a517-0f7b31b98fab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9b3557c491154848b44751ecdbc750b0', 'request_id': 'req-4da85562-4729-4b2e-a517-0f7b31b98fab'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:50.905423 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-4da85562-4729-4b2e-a517-0f7b31b98fab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9b3557c491154848b44751ecdbc750b0', 'request_id': 'req-4da85562-4729-4b2e-a517-0f7b31b98fab'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:50.931891 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e502f5f7-d7ec-4dfd-881e-c6c267d03805 service neutron] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:50.935139 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-e502f5f7-d7ec-4dfd-881e-c6c267d03805 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:51.007161 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-4da85562-4729-4b2e-a517-0f7b31b98fab admin admin] The token cache is being invalidate because project 9b3557c491154848b44751ecdbc750b0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81602) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:03:51.007161 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-4da85562-4729-4b2e-a517-0f7b31b98fab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:51.007161 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-4da85562-4729-4b2e-a517-0f7b31b98fab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:51.007161 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1252/2491] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 24 10:03:50 2022] DELETE /identity/v3/projects/9b3557c491154848b44751ecdbc750b0 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:03:51.010743 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da85562-4729-4b2e-a517-0f7b31b98fab admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:51.016087 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-4da85562-4729-4b2e-a517-0f7b31b98fab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:51.099068 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e502f5f7-d7ec-4dfd-881e-c6c267d03805 service neutron] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:51.104844 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:51.173371 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da85562-4729-4b2e-a517-0f7b31b98fab admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:51.185726 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7134cc09-562a-4eae-a087-0e13657cd3b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:51.188780 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7134cc09-562a-4eae-a087-0e13657cd3b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:51.189143 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7134cc09-562a-4eae-a087-0e13657cd3b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:51.189530 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7134cc09-562a-4eae-a087-0e13657cd3b9 admin admin] PATH_INFO: `/v3/users/79c38cb764724d1ca799d0593d0d7b1f` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:51.191016 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7134cc09-562a-4eae-a087-0e13657cd3b9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=79c38cb764724d1ca799d0593d0d7b1f)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:51.193145 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7134cc09-562a-4eae-a087-0e13657cd3b9 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:51.201968 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df13ea9-3f21-4d22-aa4c-a1b2a2bc2e54 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f055f7dc51d43bda9b315ca67f11d58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:51.202595 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df13ea9-3f21-4d22-aa4c-a1b2a2bc2e54 service nova] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:51.202873 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df13ea9-3f21-4d22-aa4c-a1b2a2bc2e54 service nova] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:51.203664 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df13ea9-3f21-4d22-aa4c-a1b2a2bc2e54 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:51.209951 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-5df13ea9-3f21-4d22-aa4c-a1b2a2bc2e54 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:51.223692 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df13ea9-3f21-4d22-aa4c-a1b2a2bc2e54 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:51.225428 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df13ea9-3f21-4d22-aa4c-a1b2a2bc2e54 service nova] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:51.236855 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1240/2492] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 10:03:50 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 10:03:51.242382 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df13ea9-3f21-4d22-aa4c-a1b2a2bc2e54 service nova] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:51.243996 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-5df13ea9-3f21-4d22-aa4c-a1b2a2bc2e54 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:51.318862 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df13ea9-3f21-4d22-aa4c-a1b2a2bc2e54 service nova] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:51.323305 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff8a01c-4568-445e-9b2e-2a8170746598 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c32b83577c46488906fe409a790b24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:51.323996 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff8a01c-4568-445e-9b2e-2a8170746598 service cinder] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:51.324278 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff8a01c-4568-445e-9b2e-2a8170746598 service cinder] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:51.324550 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff8a01c-4568-445e-9b2e-2a8170746598 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:51.336720 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-fff8a01c-4568-445e-9b2e-2a8170746598 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:51.411718 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff8a01c-4568-445e-9b2e-2a8170746598 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:51.413576 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff8a01c-4568-445e-9b2e-2a8170746598 service cinder] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:51.423841 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1241/2493] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 10:03:51 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:03:51.475476 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-7134cc09-562a-4eae-a087-0e13657cd3b9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '79c38cb764724d1ca799d0593d0d7b1f'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:51.476535 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-7134cc09-562a-4eae-a087-0e13657cd3b9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '79c38cb764724d1ca799d0593d0d7b1f'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:51.510381 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff8a01c-4568-445e-9b2e-2a8170746598 service cinder] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:51.512284 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-fff8a01c-4568-445e-9b2e-2a8170746598 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:51.535359 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-7134cc09-562a-4eae-a087-0e13657cd3b9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '79c38cb764724d1ca799d0593d0d7b1f'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:51.536386 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-7134cc09-562a-4eae-a087-0e13657cd3b9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '79c38cb764724d1ca799d0593d0d7b1f'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:51.542369 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff8a01c-4568-445e-9b2e-2a8170746598 service cinder] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:51.547831 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-7134cc09-562a-4eae-a087-0e13657cd3b9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '79c38cb764724d1ca799d0593d0d7b1f'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:51.548547 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf2b079-0527-478d-b827-e6e6c991d09d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:51.549227 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf2b079-0527-478d-b827-e6e6c991d09d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:51.549509 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf2b079-0527-478d-b827-e6e6c991d09d admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:51.549785 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf2b079-0527-478d-b827-e6e6c991d09d admin admin] PATH_INFO: `/v3/users/dcf743aadf0d47d594eb25c787302b27` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:51.550955 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf2b079-0527-478d-b827-e6e6c991d09d admin admin] RBAC: Authorizing `identity:delete_user(user_id=dcf743aadf0d47d594eb25c787302b27)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:51.553356 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf2b079-0527-478d-b827-e6e6c991d09d admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:51.581061 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1253/2494] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 24 10:03:51 2022] DELETE /identity/v3/users/79c38cb764724d1ca799d0593d0d7b1f => generated 0 bytes in 571 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:03:51.810291 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:51.824623 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-7bf2b079-0527-478d-b827-e6e6c991d09d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dcf743aadf0d47d594eb25c787302b27'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:51.826660 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-7bf2b079-0527-478d-b827-e6e6c991d09d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dcf743aadf0d47d594eb25c787302b27'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:51.846243 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-7bf2b079-0527-478d-b827-e6e6c991d09d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dcf743aadf0d47d594eb25c787302b27'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:51.854410 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-7bf2b079-0527-478d-b827-e6e6c991d09d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dcf743aadf0d47d594eb25c787302b27'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:51.950511 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-7bf2b079-0527-478d-b827-e6e6c991d09d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dcf743aadf0d47d594eb25c787302b27'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:51.962383 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1242/2495] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 24 10:03:51 2022] DELETE /identity/v3/users/dcf743aadf0d47d594eb25c787302b27 => generated 0 bytes in 452 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 10:03:51.966935 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d57495d-2bef-4cfb-94bb-537513e7098a None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:51.967233 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d57495d-2bef-4cfb-94bb-537513e7098a None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:51.967504 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d57495d-2bef-4cfb-94bb-537513e7098a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:51.989425 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-7d57495d-2bef-4cfb-94bb-537513e7098a None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:03:51.990642 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-7d57495d-2bef-4cfb-94bb-537513e7098a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:52.045235 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:52.056924 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1254/2496] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 10:03:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:03:52.067081 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf2b079-0527-478d-b827-e6e6c991d09d admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:52.068622 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-7bf2b079-0527-478d-b827-e6e6c991d09d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:52.099847 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d57495d-2bef-4cfb-94bb-537513e7098a None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:52.101652 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-7d57495d-2bef-4cfb-94bb-537513e7098a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:52.118842 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d57495d-2bef-4cfb-94bb-537513e7098a None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:52.126551 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0dc407-6c07-4a2e-add2-490c0a82fadb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:52.127548 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0dc407-6c07-4a2e-add2-490c0a82fadb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:52.127900 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0dc407-6c07-4a2e-add2-490c0a82fadb admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:52.129561 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0dc407-6c07-4a2e-add2-490c0a82fadb admin admin] PATH_INFO: `/v3/projects/71fd88e02e834acabd1f8c9e660ff869` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:52.131169 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0dc407-6c07-4a2e-add2-490c0a82fadb admin admin] RBAC: Authorizing `identity:delete_project(project_id=71fd88e02e834acabd1f8c9e660ff869)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:52.133517 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0dc407-6c07-4a2e-add2-490c0a82fadb admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:52.148947 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf2b079-0527-478d-b827-e6e6c991d09d admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:52.153647 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14981419-2638-41d3-b02c-0b6fd866b71b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e5260e798404e8bb12b25ca8ea97052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:52.154560 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14981419-2638-41d3-b02c-0b6fd866b71b service neutron] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:52.155198 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14981419-2638-41d3-b02c-0b6fd866b71b service neutron] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:52.155730 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14981419-2638-41d3-b02c-0b6fd866b71b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:52.167646 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14981419-2638-41d3-b02c-0b6fd866b71b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:52.169537 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14981419-2638-41d3-b02c-0b6fd866b71b service neutron] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:52.188470 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1243/2497] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 10:03:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:03:52.243634 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-0f0dc407-6c07-4a2e-add2-490c0a82fadb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '71fd88e02e834acabd1f8c9e660ff869', 'request_id': 'req-0f0dc407-6c07-4a2e-add2-490c0a82fadb'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:52.249983 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-0f0dc407-6c07-4a2e-add2-490c0a82fadb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '71fd88e02e834acabd1f8c9e660ff869', 'request_id': 'req-0f0dc407-6c07-4a2e-add2-490c0a82fadb'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:52.255595 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-0f0dc407-6c07-4a2e-add2-490c0a82fadb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '71fd88e02e834acabd1f8c9e660ff869', 'request_id': 'req-0f0dc407-6c07-4a2e-add2-490c0a82fadb'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:52.370224 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-0f0dc407-6c07-4a2e-add2-490c0a82fadb admin admin] The token cache is being invalidate because project 71fd88e02e834acabd1f8c9e660ff869 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81602) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:03:52.370929 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-0f0dc407-6c07-4a2e-add2-490c0a82fadb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:52.371526 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-0f0dc407-6c07-4a2e-add2-490c0a82fadb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:52.373564 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1255/2498] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 24 10:03:52 2022] DELETE /identity/v3/projects/71fd88e02e834acabd1f8c9e660ff869 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:03:52.378235 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2d097b-c325-461d-b310-03dba3045374 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:52.378585 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2d097b-c325-461d-b310-03dba3045374 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:52.378777 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2d097b-c325-461d-b310-03dba3045374 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:52.399431 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-1c2d097b-c325-461d-b310-03dba3045374 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:03:52.401024 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-1c2d097b-c325-461d-b310-03dba3045374 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:52.468808 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1244/2499] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 10:03:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:03:52.476865 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d1950e-d13e-4fdf-8e27-c839d13b5e39 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:52.477330 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d1950e-d13e-4fdf-8e27-c839d13b5e39 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:52.477676 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d1950e-d13e-4fdf-8e27-c839d13b5e39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:52.497837 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-b4d1950e-d13e-4fdf-8e27-c839d13b5e39 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:03:52.499100 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-b4d1950e-d13e-4fdf-8e27-c839d13b5e39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:52.507698 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1256/2500] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 10:03:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:03:52.512087 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2d097b-c325-461d-b310-03dba3045374 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:52.522420 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2d097b-c325-461d-b310-03dba3045374 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:52.528116 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5a41d2-f81c-4e67-8c1a-5d3ef0363f26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:52.528116 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5a41d2-f81c-4e67-8c1a-5d3ef0363f26 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:52.534204 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5a41d2-f81c-4e67-8c1a-5d3ef0363f26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:52.534204 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5a41d2-f81c-4e67-8c1a-5d3ef0363f26 admin admin] PATH_INFO: `/v3/domains` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:52.534204 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5a41d2-f81c-4e67-8c1a-5d3ef0363f26 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:03:52.534204 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5a41d2-f81c-4e67-8c1a-5d3ef0363f26 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:52.534204 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5a41d2-f81c-4e67-8c1a-5d3ef0363f26 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:52.547756 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1245/2501] 10.222.0.16 () {58 vars in 1153 bytes} [Wed Aug 24 10:03:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:03:52.550342 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d1950e-d13e-4fdf-8e27-c839d13b5e39 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:52.560662 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d1950e-d13e-4fdf-8e27-c839d13b5e39 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:52.565612 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814ff328-86c4-45e0-97d2-65c48a1c9019 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:52.568060 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814ff328-86c4-45e0-97d2-65c48a1c9019 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:52.568060 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814ff328-86c4-45e0-97d2-65c48a1c9019 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:52.568060 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814ff328-86c4-45e0-97d2-65c48a1c9019 admin admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:52.568060 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814ff328-86c4-45e0-97d2-65c48a1c9019 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:52.570446 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814ff328-86c4-45e0-97d2-65c48a1c9019 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:52.600832 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5a41d2-f81c-4e67-8c1a-5d3ef0363f26 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:52.602663 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-4d5a41d2-f81c-4e67-8c1a-5d3ef0363f26 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:52.617128 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5a41d2-f81c-4e67-8c1a-5d3ef0363f26 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:52.621865 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a68969-65ac-4ebf-ba46-858683f40ff4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:52.622504 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a68969-65ac-4ebf-ba46-858683f40ff4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:52.622784 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a68969-65ac-4ebf-ba46-858683f40ff4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:52.623056 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a68969-65ac-4ebf-ba46-858683f40ff4 admin admin] PATH_INFO: `/v3/projects/4556da2c51c84e2d85c823d4041d0171` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:52.624196 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a68969-65ac-4ebf-ba46-858683f40ff4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4556da2c51c84e2d85c823d4041d0171)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:52.626272 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a68969-65ac-4ebf-ba46-858683f40ff4 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:52.682141 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1257/2502] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 24 10:03:52 2022] POST /identity/v3/projects => generated 363 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:03:52.683330 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814ff328-86c4-45e0-97d2-65c48a1c9019 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:52.748462 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:52.752615 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-97a68969-65ac-4ebf-ba46-858683f40ff4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4556da2c51c84e2d85c823d4041d0171', 'request_id': 'req-97a68969-65ac-4ebf-ba46-858683f40ff4'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:52.752827 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814ff328-86c4-45e0-97d2-65c48a1c9019 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:52.760787 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-97a68969-65ac-4ebf-ba46-858683f40ff4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4556da2c51c84e2d85c823d4041d0171', 'request_id': 'req-97a68969-65ac-4ebf-ba46-858683f40ff4'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:52.770918 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-97a68969-65ac-4ebf-ba46-858683f40ff4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4556da2c51c84e2d85c823d4041d0171', 'request_id': 'req-97a68969-65ac-4ebf-ba46-858683f40ff4'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:52.820055 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfacbc1e-7695-4eae-9646-6a46e2f0a374 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:52.820671 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfacbc1e-7695-4eae-9646-6a46e2f0a374 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:52.820948 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfacbc1e-7695-4eae-9646-6a46e2f0a374 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:52.821221 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfacbc1e-7695-4eae-9646-6a46e2f0a374 admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:52.821839 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfacbc1e-7695-4eae-9646-6a46e2f0a374 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:52.823766 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfacbc1e-7695-4eae-9646-6a46e2f0a374 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:52.842649 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:52.847564 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-97a68969-65ac-4ebf-ba46-858683f40ff4 admin admin] The token cache is being invalidate because project 4556da2c51c84e2d85c823d4041d0171 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:03:52.847949 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-97a68969-65ac-4ebf-ba46-858683f40ff4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:52.848594 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-97a68969-65ac-4ebf-ba46-858683f40ff4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:52.850072 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1246/2503] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 24 10:03:52 2022] DELETE /identity/v3/projects/4556da2c51c84e2d85c823d4041d0171 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 10:03:52.853914 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a68969-65ac-4ebf-ba46-858683f40ff4 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:52.855223 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-97a68969-65ac-4ebf-ba46-858683f40ff4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:52.868513 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a68969-65ac-4ebf-ba46-858683f40ff4 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:52.875749 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45de9cec-b197-43b3-8864-8a205e9b8a13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:52.876403 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45de9cec-b197-43b3-8864-8a205e9b8a13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:52.876688 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45de9cec-b197-43b3-8864-8a205e9b8a13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:52.877503 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45de9cec-b197-43b3-8864-8a205e9b8a13 admin admin] PATH_INFO: `/v3/users/8f8b34867b9a4499b5c6aa77885eb6bd` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:52.879185 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45de9cec-b197-43b3-8864-8a205e9b8a13 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8f8b34867b9a4499b5c6aa77885eb6bd)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:52.881589 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45de9cec-b197-43b3-8864-8a205e9b8a13 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:52.913216 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1258/2504] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 24 10:03:52 2022] POST /identity/v3/users => generated 282 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:03:52.914224 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfacbc1e-7695-4eae-9646-6a46e2f0a374 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:52.916817 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-bfacbc1e-7695-4eae-9646-6a46e2f0a374 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:52.929478 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfacbc1e-7695-4eae-9646-6a46e2f0a374 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:52.934223 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d9e492-c921-47fd-aa0d-efc4e8cf6b29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:52.934833 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d9e492-c921-47fd-aa0d-efc4e8cf6b29 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:52.935118 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d9e492-c921-47fd-aa0d-efc4e8cf6b29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:52.935389 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d9e492-c921-47fd-aa0d-efc4e8cf6b29 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:52.936090 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d9e492-c921-47fd-aa0d-efc4e8cf6b29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:52.937732 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d9e492-c921-47fd-aa0d-efc4e8cf6b29 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:52.950016 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1259/2505] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 24 10:03:52 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 10:03:52.953664 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d9e492-c921-47fd-aa0d-efc4e8cf6b29 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:52.965707 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d9e492-c921-47fd-aa0d-efc4e8cf6b29 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:52.971127 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe70a571-08b5-44a3-b0fa-4bd665d290ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:52.972449 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe70a571-08b5-44a3-b0fa-4bd665d290ce admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:52.972594 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe70a571-08b5-44a3-b0fa-4bd665d290ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:52.972873 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe70a571-08b5-44a3-b0fa-4bd665d290ce admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:52.973523 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe70a571-08b5-44a3-b0fa-4bd665d290ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:52.975116 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe70a571-08b5-44a3-b0fa-4bd665d290ce admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:52.994217 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1260/2506] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 24 10:03:52 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 10:03:52.998528 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe70a571-08b5-44a3-b0fa-4bd665d290ce admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:53.011329 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe70a571-08b5-44a3-b0fa-4bd665d290ce admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:53.018252 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d36db56-1e42-4b81-94cb-63fe7201e0c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:53.018252 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d36db56-1e42-4b81-94cb-63fe7201e0c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:53.018252 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d36db56-1e42-4b81-94cb-63fe7201e0c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:53.018662 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d36db56-1e42-4b81-94cb-63fe7201e0c2 admin admin] PATH_INFO: `/v3/projects/caf4ada5edb94caf8f918b16599609ae/users/94e9fefaad6a425a943832bb9555d6fc/roles/136cdf78e00a426997dd4e37faff489b` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:53.052974 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d36db56-1e42-4b81-94cb-63fe7201e0c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=caf4ada5edb94caf8f918b16599609ae, user_id=94e9fefaad6a425a943832bb9555d6fc, role_id=136cdf78e00a426997dd4e37faff489b)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:53.052974 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d36db56-1e42-4b81-94cb-63fe7201e0c2 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:53.155917 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-45de9cec-b197-43b3-8864-8a205e9b8a13 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8f8b34867b9a4499b5c6aa77885eb6bd'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:53.157118 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-45de9cec-b197-43b3-8864-8a205e9b8a13 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8f8b34867b9a4499b5c6aa77885eb6bd'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:53.172492 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-45de9cec-b197-43b3-8864-8a205e9b8a13 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8f8b34867b9a4499b5c6aa77885eb6bd'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:53.186585 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-45de9cec-b197-43b3-8864-8a205e9b8a13 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8f8b34867b9a4499b5c6aa77885eb6bd'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:53.195406 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1261/2507] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 24 10:03:52 2022] PUT /identity/v3/projects/caf4ada5edb94caf8f918b16599609ae/users/94e9fefaad6a425a943832bb9555d6fc/roles/136cdf78e00a426997dd4e37faff489b => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 10:03:53.197336 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f88e8e9-8c3c-4aa0-b4ee-26b198cd311c None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:53.197887 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f88e8e9-8c3c-4aa0-b4ee-26b198cd311c None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:53.197887 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f88e8e9-8c3c-4aa0-b4ee-26b198cd311c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:53.226529 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-3f88e8e9-8c3c-4aa0-b4ee-26b198cd311c None None] MFA Rules not processed for user `94e9fefaad6a425a943832bb9555d6fc`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:03:53.227847 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-3f88e8e9-8c3c-4aa0-b4ee-26b198cd311c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:53.291470 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1262/2508] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 10:03:53 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:03:53.296147 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f88e8e9-8c3c-4aa0-b4ee-26b198cd311c None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:53.296313 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-45de9cec-b197-43b3-8864-8a205e9b8a13 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8f8b34867b9a4499b5c6aa77885eb6bd'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:53.306237 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1247/2509] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 24 10:03:52 2022] DELETE /identity/v3/users/8f8b34867b9a4499b5c6aa77885eb6bd => generated 0 bytes in 453 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 10:03:53.365495 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f88e8e9-8c3c-4aa0-b4ee-26b198cd311c None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:53.447775 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09b7b58-ca56-45b4-b12f-c68efb437329 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:53.448996 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09b7b58-ca56-45b4-b12f-c68efb437329 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:53.449289 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09b7b58-ca56-45b4-b12f-c68efb437329 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:53.449790 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09b7b58-ca56-45b4-b12f-c68efb437329 admin admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:53.450840 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09b7b58-ca56-45b4-b12f-c68efb437329 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:53.454778 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09b7b58-ca56-45b4-b12f-c68efb437329 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:53.537923 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1263/2510] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 24 10:03:53 2022] POST /identity/v3/projects => generated 365 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 10:03:53.542991 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45de9cec-b197-43b3-8864-8a205e9b8a13 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:53.544938 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-45de9cec-b197-43b3-8864-8a205e9b8a13 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:53.560170 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45de9cec-b197-43b3-8864-8a205e9b8a13 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:53.641525 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf90b0b3-5f3c-4dea-9140-8afc87bc52e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:53.642235 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf90b0b3-5f3c-4dea-9140-8afc87bc52e7 admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:53.642625 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf90b0b3-5f3c-4dea-9140-8afc87bc52e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:53.642984 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf90b0b3-5f3c-4dea-9140-8afc87bc52e7 admin admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:53.643625 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf90b0b3-5f3c-4dea-9140-8afc87bc52e7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:53.648609 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf90b0b3-5f3c-4dea-9140-8afc87bc52e7 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:53.749032 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1248/2511] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 24 10:03:53 2022] POST /identity/v3/users => generated 289 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:03:53.755036 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09b7b58-ca56-45b4-b12f-c68efb437329 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:53.765929 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09b7b58-ca56-45b4-b12f-c68efb437329 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:53.770713 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270c385f-c9fa-4a52-a251-24068d270c59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:53.771327 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270c385f-c9fa-4a52-a251-24068d270c59 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:53.775019 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270c385f-c9fa-4a52-a251-24068d270c59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:53.775019 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270c385f-c9fa-4a52-a251-24068d270c59 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:53.775019 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270c385f-c9fa-4a52-a251-24068d270c59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:53.775019 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270c385f-c9fa-4a52-a251-24068d270c59 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:53.787163 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1264/2512] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 24 10:03:53 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:03:53.791459 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf90b0b3-5f3c-4dea-9140-8afc87bc52e7 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:53.802685 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf90b0b3-5f3c-4dea-9140-8afc87bc52e7 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:53.807669 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8018f05a-4321-410f-8c6b-9f2798a4df07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:53.808597 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8018f05a-4321-410f-8c6b-9f2798a4df07 admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:53.808883 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8018f05a-4321-410f-8c6b-9f2798a4df07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:53.809220 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8018f05a-4321-410f-8c6b-9f2798a4df07 admin admin] PATH_INFO: `/v3/projects/b10250b5d4c647c1a2c42544aeb0b2ef/users/9b0fcdf55c80405da239f221af9805c6/roles/15f634d4a2d247ddab90c458e5e1c141` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:53.824121 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8018f05a-4321-410f-8c6b-9f2798a4df07 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b10250b5d4c647c1a2c42544aeb0b2ef, user_id=9b0fcdf55c80405da239f221af9805c6, role_id=15f634d4a2d247ddab90c458e5e1c141)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:53.826823 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8018f05a-4321-410f-8c6b-9f2798a4df07 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:53.856646 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270c385f-c9fa-4a52-a251-24068d270c59 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:53.858386 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-270c385f-c9fa-4a52-a251-24068d270c59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:53.877372 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270c385f-c9fa-4a52-a251-24068d270c59 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:53.883403 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b436d94b-ae25-4445-8713-e92d772a2b66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:53.884194 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b436d94b-ae25-4445-8713-e92d772a2b66 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:53.884550 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b436d94b-ae25-4445-8713-e92d772a2b66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:53.884898 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b436d94b-ae25-4445-8713-e92d772a2b66 admin admin] PATH_INFO: `/v3/projects/a5d90d6bc57e452aa5b840578e7becf0` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:53.886086 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b436d94b-ae25-4445-8713-e92d772a2b66 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a5d90d6bc57e452aa5b840578e7becf0)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:53.888187 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b436d94b-ae25-4445-8713-e92d772a2b66 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:53.960477 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1249/2513] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 24 10:03:53 2022] PUT /identity/v3/projects/b10250b5d4c647c1a2c42544aeb0b2ef/users/9b0fcdf55c80405da239f221af9805c6/roles/15f634d4a2d247ddab90c458e5e1c141 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:03:53.964429 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8018f05a-4321-410f-8c6b-9f2798a4df07 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:53.977197 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:DELETE=14,keystone:SELECT=80,keystone:INSERT=6 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:54.025778 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8018f05a-4321-410f-8c6b-9f2798a4df07 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:54.027928 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-b436d94b-ae25-4445-8713-e92d772a2b66 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a5d90d6bc57e452aa5b840578e7becf0', 'request_id': 'req-b436d94b-ae25-4445-8713-e92d772a2b66'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:54.034700 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-b436d94b-ae25-4445-8713-e92d772a2b66 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a5d90d6bc57e452aa5b840578e7becf0', 'request_id': 'req-b436d94b-ae25-4445-8713-e92d772a2b66'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:54.040361 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-b436d94b-ae25-4445-8713-e92d772a2b66 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a5d90d6bc57e452aa5b840578e7becf0', 'request_id': 'req-b436d94b-ae25-4445-8713-e92d772a2b66'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:54.084600 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f166cf08-5b83-4be7-b866-4bc82dea67c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:54.085260 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f166cf08-5b83-4be7-b866-4bc82dea67c3 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:54.085550 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f166cf08-5b83-4be7-b866-4bc82dea67c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:54.085826 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f166cf08-5b83-4be7-b866-4bc82dea67c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:54.086476 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f166cf08-5b83-4be7-b866-4bc82dea67c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:54.088466 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f166cf08-5b83-4be7-b866-4bc82dea67c3 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:54.099828 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1250/2514] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 24 10:03:53 2022] GET /identity/v3/roles => generated 1420 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 10:03:54.103684 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f166cf08-5b83-4be7-b866-4bc82dea67c3 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:54.113744 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f166cf08-5b83-4be7-b866-4bc82dea67c3 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:54.119419 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb404c5-bc8f-4a4c-be51-751d17de2923 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:54.120124 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb404c5-bc8f-4a4c-be51-751d17de2923 admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:54.120406 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb404c5-bc8f-4a4c-be51-751d17de2923 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:54.120678 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb404c5-bc8f-4a4c-be51-751d17de2923 admin admin] PATH_INFO: `/v3/projects/b10250b5d4c647c1a2c42544aeb0b2ef/users/9b0fcdf55c80405da239f221af9805c6/roles/15f634d4a2d247ddab90c458e5e1c141` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:54.122786 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb404c5-bc8f-4a4c-be51-751d17de2923 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b10250b5d4c647c1a2c42544aeb0b2ef, user_id=9b0fcdf55c80405da239f221af9805c6, role_id=15f634d4a2d247ddab90c458e5e1c141)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:54.135251 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-b436d94b-ae25-4445-8713-e92d772a2b66 admin admin] The token cache is being invalidate because project a5d90d6bc57e452aa5b840578e7becf0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81602) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:03:54.135251 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-b436d94b-ae25-4445-8713-e92d772a2b66 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:54.135251 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-b436d94b-ae25-4445-8713-e92d772a2b66 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:54.135530 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb404c5-bc8f-4a4c-be51-751d17de2923 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:54.135662 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1265/2515] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 24 10:03:53 2022] DELETE /identity/v3/projects/a5d90d6bc57e452aa5b840578e7becf0 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:03:54.136132 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1251/2516] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 24 10:03:54 2022] PUT /identity/v3/projects/b10250b5d4c647c1a2c42544aeb0b2ef/users/9b0fcdf55c80405da239f221af9805c6/roles/15f634d4a2d247ddab90c458e5e1c141 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:03:54.136666 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3e354b-a18a-4e6d-ab5a-7e1b6f53226d None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:54.136951 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3e354b-a18a-4e6d-ab5a-7e1b6f53226d None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:54.137228 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3e354b-a18a-4e6d-ab5a-7e1b6f53226d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:54.146053 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ce5824-1377-4c71-9f06-ba4898d9ed3f None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:54.146418 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ce5824-1377-4c71-9f06-ba4898d9ed3f None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:54.146692 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ce5824-1377-4c71-9f06-ba4898d9ed3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:54.157423 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-5b3e354b-a18a-4e6d-ab5a-7e1b6f53226d None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:03:54.159185 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-5b3e354b-a18a-4e6d-ab5a-7e1b6f53226d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:54.174663 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-d4ce5824-1377-4c71-9f06-ba4898d9ed3f None None] MFA Rules not processed for user `9b0fcdf55c80405da239f221af9805c6`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:03:54.175934 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-d4ce5824-1377-4c71-9f06-ba4898d9ed3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:54.188442 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:54.225341 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1266/2517] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 10:03:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:03:54.233338 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206c7669-d616-4a9f-be35-d94065cee2db None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:54.233622 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206c7669-d616-4a9f-be35-d94065cee2db None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:54.233897 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206c7669-d616-4a9f-be35-d94065cee2db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:54.253661 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-206c7669-d616-4a9f-be35-d94065cee2db None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:03:54.255065 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-206c7669-d616-4a9f-be35-d94065cee2db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:54.257078 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1252/2518] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 10:03:54 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:03:54.267277 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1267/2519] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 10:03:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:03:54.270927 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ce5824-1377-4c71-9f06-ba4898d9ed3f None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:54.271075 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-d4ce5824-1377-4c71-9f06-ba4898d9ed3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:54.271909 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206c7669-d616-4a9f-be35-d94065cee2db None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:54.281396 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206c7669-d616-4a9f-be35-d94065cee2db None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:54.286818 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8082b38-56b0-4718-9737-73ac0831543e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:54.287419 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8082b38-56b0-4718-9737-73ac0831543e admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:54.287799 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8082b38-56b0-4718-9737-73ac0831543e admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:54.288121 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8082b38-56b0-4718-9737-73ac0831543e admin admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:54.288812 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8082b38-56b0-4718-9737-73ac0831543e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:03:54.289117 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8082b38-56b0-4718-9737-73ac0831543e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:54.290735 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8082b38-56b0-4718-9737-73ac0831543e admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:54.303726 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1268/2520] 10.222.0.16 () {58 vars in 1153 bytes} [Wed Aug 24 10:03:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:03:54.307612 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8082b38-56b0-4718-9737-73ac0831543e admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:54.319067 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8082b38-56b0-4718-9737-73ac0831543e admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:54.324497 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c7238b-e073-4684-bb56-35bbc46d91c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:54.325377 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c7238b-e073-4684-bb56-35bbc46d91c5 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:54.325587 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c7238b-e073-4684-bb56-35bbc46d91c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:54.325862 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c7238b-e073-4684-bb56-35bbc46d91c5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:54.326544 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c7238b-e073-4684-bb56-35bbc46d91c5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:54.328634 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c7238b-e073-4684-bb56-35bbc46d91c5 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:54.341319 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ce5824-1377-4c71-9f06-ba4898d9ed3f None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:54.346225 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf01e4f4-42f4-4735-a935-c9728ca1d5d2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c32b83577c46488906fe409a790b24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:54.346897 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf01e4f4-42f4-4735-a935-c9728ca1d5d2 service cinder] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:54.347176 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf01e4f4-42f4-4735-a935-c9728ca1d5d2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:54.347448 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf01e4f4-42f4-4735-a935-c9728ca1d5d2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:54.358164 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf01e4f4-42f4-4735-a935-c9728ca1d5d2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:54.359790 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf01e4f4-42f4-4735-a935-c9728ca1d5d2 service cinder] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:54.370019 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1253/2521] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 10:03:54 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:03:54.477706 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1269/2522] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 24 10:03:54 2022] POST /identity/v3/projects => generated 391 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:03:54.480259 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf01e4f4-42f4-4735-a935-c9728ca1d5d2 service cinder] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:54.497376 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf01e4f4-42f4-4735-a935-c9728ca1d5d2 service cinder] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:54.583442 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631b784a-8ad4-494c-90e6-fa21767014fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:54.584399 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631b784a-8ad4-494c-90e6-fa21767014fd admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:54.584747 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631b784a-8ad4-494c-90e6-fa21767014fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:54.585088 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631b784a-8ad4-494c-90e6-fa21767014fd admin admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:54.585786 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631b784a-8ad4-494c-90e6-fa21767014fd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:54.587907 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631b784a-8ad4-494c-90e6-fa21767014fd admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:54.700576 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1254/2523] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 24 10:03:54 2022] POST /identity/v3/users => generated 296 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 10:03:54.704948 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c7238b-e073-4684-bb56-35bbc46d91c5 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:54.715441 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c7238b-e073-4684-bb56-35bbc46d91c5 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:54.720177 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0772793-05e0-4eab-a837-03552809ce51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:54.720764 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0772793-05e0-4eab-a837-03552809ce51 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:54.721652 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0772793-05e0-4eab-a837-03552809ce51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:54.721925 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0772793-05e0-4eab-a837-03552809ce51 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:54.722566 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0772793-05e0-4eab-a837-03552809ce51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:54.724172 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0772793-05e0-4eab-a837-03552809ce51 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:54.734449 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1270/2524] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 24 10:03:54 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:03:54.739724 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631b784a-8ad4-494c-90e6-fa21767014fd admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:54.749835 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631b784a-8ad4-494c-90e6-fa21767014fd admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:54.754613 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e7b984-2359-4e23-a86a-61778b02e34a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:54.755428 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e7b984-2359-4e23-a86a-61778b02e34a admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:54.755688 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e7b984-2359-4e23-a86a-61778b02e34a admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:54.756036 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e7b984-2359-4e23-a86a-61778b02e34a admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:54.756800 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e7b984-2359-4e23-a86a-61778b02e34a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:54.758509 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e7b984-2359-4e23-a86a-61778b02e34a admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:54.769107 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1255/2525] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 24 10:03:54 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:03:54.773280 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0772793-05e0-4eab-a837-03552809ce51 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:54.783210 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0772793-05e0-4eab-a837-03552809ce51 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:54.788656 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7380fb2-00b4-453b-a7f9-3cd053ca562f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:54.789305 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7380fb2-00b4-453b-a7f9-3cd053ca562f admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:54.789586 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7380fb2-00b4-453b-a7f9-3cd053ca562f admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:54.789856 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7380fb2-00b4-453b-a7f9-3cd053ca562f admin admin] PATH_INFO: `/v3/projects/160be357dd814b05951273708265833f/users/5d45e9fcb8244b4ab91dfd251ae84216/roles/136cdf78e00a426997dd4e37faff489b` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:54.804843 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7380fb2-00b4-453b-a7f9-3cd053ca562f admin admin] RBAC: Authorizing `identity:create_grant(project_id=160be357dd814b05951273708265833f, user_id=5d45e9fcb8244b4ab91dfd251ae84216, role_id=136cdf78e00a426997dd4e37faff489b)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:54.808185 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7380fb2-00b4-453b-a7f9-3cd053ca562f admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:54.887347 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1271/2526] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 24 10:03:54 2022] PUT /identity/v3/projects/160be357dd814b05951273708265833f/users/5d45e9fcb8244b4ab91dfd251ae84216/roles/136cdf78e00a426997dd4e37faff489b => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:03:54.898037 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83092434-0144-4e43-929e-f520dc1c0423 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:54.898320 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83092434-0144-4e43-929e-f520dc1c0423 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:54.898593 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83092434-0144-4e43-929e-f520dc1c0423 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:54.926753 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-83092434-0144-4e43-929e-f520dc1c0423 None None] MFA Rules not processed for user `5d45e9fcb8244b4ab91dfd251ae84216`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:03:54.928103 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-83092434-0144-4e43-929e-f520dc1c0423 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:55.001798 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1256/2527] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 10:03:54 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:03:55.182401 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7380fb2-00b4-453b-a7f9-3cd053ca562f admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:55.188717 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:55.245948 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7380fb2-00b4-453b-a7f9-3cd053ca562f admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:55.256173 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46710c9b-d83d-456a-aab4-9d907b42edb3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c32b83577c46488906fe409a790b24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:55.256173 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46710c9b-d83d-456a-aab4-9d907b42edb3 service cinder] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:55.256173 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46710c9b-d83d-456a-aab4-9d907b42edb3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:55.256173 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46710c9b-d83d-456a-aab4-9d907b42edb3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:55.264194 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46710c9b-d83d-456a-aab4-9d907b42edb3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:55.265808 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46710c9b-d83d-456a-aab4-9d907b42edb3 service cinder] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:55.276202 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1272/2528] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 10:03:55 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:03:55.576193 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:56.069890 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83092434-0144-4e43-929e-f520dc1c0423 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:56.071343 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-83092434-0144-4e43-929e-f520dc1c0423 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:56.151132 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83092434-0144-4e43-929e-f520dc1c0423 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:56.155935 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49de0a95-8ba8-40ce-8938-06378eed6ab6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:56.156682 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49de0a95-8ba8-40ce-8938-06378eed6ab6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:56.156852 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49de0a95-8ba8-40ce-8938-06378eed6ab6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:56.157122 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49de0a95-8ba8-40ce-8938-06378eed6ab6 admin admin] PATH_INFO: `/v3/users/bfec0f7f68b84818a24c939f250fc067` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:56.158258 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49de0a95-8ba8-40ce-8938-06378eed6ab6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bfec0f7f68b84818a24c939f250fc067)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:56.160725 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49de0a95-8ba8-40ce-8938-06378eed6ab6 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:56.202903 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38690b8-1973-4f92-802a-9c96dae5ab0e None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:56.203735 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38690b8-1973-4f92-802a-9c96dae5ab0e None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:56.204463 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38690b8-1973-4f92-802a-9c96dae5ab0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:56.235118 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-d38690b8-1973-4f92-802a-9c96dae5ab0e None None] MFA Rules not processed for user `def04335e57e48409a6943fa231d47c3`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:03:56.236572 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-d38690b8-1973-4f92-802a-9c96dae5ab0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:56.302955 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1273/2529] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 10:03:56 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:03:56.304060 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38690b8-1973-4f92-802a-9c96dae5ab0e None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:56.308553 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-d38690b8-1973-4f92-802a-9c96dae5ab0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:56.322351 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38690b8-1973-4f92-802a-9c96dae5ab0e None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:56.326954 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab65ac6-954f-421b-8ae9-69be5ee3c95e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:56.327614 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab65ac6-954f-421b-8ae9-69be5ee3c95e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:56.327893 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab65ac6-954f-421b-8ae9-69be5ee3c95e admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:56.328165 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab65ac6-954f-421b-8ae9-69be5ee3c95e admin admin] PATH_INFO: `/v3/users/94e9fefaad6a425a943832bb9555d6fc` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:56.329804 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab65ac6-954f-421b-8ae9-69be5ee3c95e admin admin] RBAC: Authorizing `identity:delete_user(user_id=94e9fefaad6a425a943832bb9555d6fc)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:56.332351 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab65ac6-954f-421b-8ae9-69be5ee3c95e admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:56.372205 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-49de0a95-8ba8-40ce-8938-06378eed6ab6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bfec0f7f68b84818a24c939f250fc067'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:56.372895 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-49de0a95-8ba8-40ce-8938-06378eed6ab6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bfec0f7f68b84818a24c939f250fc067'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:56.384880 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-49de0a95-8ba8-40ce-8938-06378eed6ab6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bfec0f7f68b84818a24c939f250fc067'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:56.394285 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-49de0a95-8ba8-40ce-8938-06378eed6ab6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bfec0f7f68b84818a24c939f250fc067'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:56.473019 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-49de0a95-8ba8-40ce-8938-06378eed6ab6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bfec0f7f68b84818a24c939f250fc067'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:56.479405 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1257/2530] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 24 10:03:56 2022] DELETE /identity/v3/users/bfec0f7f68b84818a24c939f250fc067 => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:03:56.484369 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b80d673-8256-45fe-84cb-9d0aaf099331 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:56.485062 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b80d673-8256-45fe-84cb-9d0aaf099331 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:56.486800 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b80d673-8256-45fe-84cb-9d0aaf099331 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:56.490092 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:56.497463 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-8ab65ac6-954f-421b-8ae9-69be5ee3c95e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '94e9fefaad6a425a943832bb9555d6fc'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:56.498380 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-8ab65ac6-954f-421b-8ae9-69be5ee3c95e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '94e9fefaad6a425a943832bb9555d6fc'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:56.507710 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-9b80d673-8256-45fe-84cb-9d0aaf099331 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:03:56.511715 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-9b80d673-8256-45fe-84cb-9d0aaf099331 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:56.558164 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:56.569557 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1258/2531] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 10:03:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:03:56.578768 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b80d673-8256-45fe-84cb-9d0aaf099331 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:56.581137 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-9b80d673-8256-45fe-84cb-9d0aaf099331 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:56.581691 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-8ab65ac6-954f-421b-8ae9-69be5ee3c95e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '94e9fefaad6a425a943832bb9555d6fc'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:56.582341 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-8ab65ac6-954f-421b-8ae9-69be5ee3c95e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '94e9fefaad6a425a943832bb9555d6fc'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:56.590715 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-8ab65ac6-954f-421b-8ae9-69be5ee3c95e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '94e9fefaad6a425a943832bb9555d6fc'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:56.608360 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1274/2532] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 24 10:03:56 2022] DELETE /identity/v3/users/94e9fefaad6a425a943832bb9555d6fc => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:03:56.613285 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efb9456-0f1b-4068-9c36-6d92c6d368b7 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:56.613641 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efb9456-0f1b-4068-9c36-6d92c6d368b7 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:56.613990 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efb9456-0f1b-4068-9c36-6d92c6d368b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:56.644546 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b80d673-8256-45fe-84cb-9d0aaf099331 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:56.648235 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-4efb9456-0f1b-4068-9c36-6d92c6d368b7 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:03:56.649417 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-4efb9456-0f1b-4068-9c36-6d92c6d368b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:56.694243 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff377e9-b318-4c86-bf0d-883a71f02aac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e5260e798404e8bb12b25ca8ea97052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:56.694855 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff377e9-b318-4c86-bf0d-883a71f02aac service neutron] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:56.695132 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff377e9-b318-4c86-bf0d-883a71f02aac service neutron] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:56.695400 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff377e9-b318-4c86-bf0d-883a71f02aac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:56.697174 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-aff377e9-b318-4c86-bf0d-883a71f02aac service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:56.709002 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1275/2533] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 10:03:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:03:56.712582 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efb9456-0f1b-4068-9c36-6d92c6d368b7 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:56.713989 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-4efb9456-0f1b-4068-9c36-6d92c6d368b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:56.725941 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efb9456-0f1b-4068-9c36-6d92c6d368b7 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:56.729493 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269c4045-351f-4901-8873-460917fe0c22 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e5260e798404e8bb12b25ca8ea97052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:56.730086 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269c4045-351f-4901-8873-460917fe0c22 service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:56.730360 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269c4045-351f-4901-8873-460917fe0c22 service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:56.730627 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269c4045-351f-4901-8873-460917fe0c22 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:56.732410 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-269c4045-351f-4901-8873-460917fe0c22 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:56.741727 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269c4045-351f-4901-8873-460917fe0c22 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:56.743139 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269c4045-351f-4901-8873-460917fe0c22 service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:56.751052 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1276/2534] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 10:03:56 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:03:56.753110 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269c4045-351f-4901-8873-460917fe0c22 service neutron] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:56.761601 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269c4045-351f-4901-8873-460917fe0c22 service neutron] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:56.762726 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff377e9-b318-4c86-bf0d-883a71f02aac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:56.764238 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff377e9-b318-4c86-bf0d-883a71f02aac service neutron] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:56.765333 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5980746-d6b2-44d7-ad62-203d48f68990 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e5260e798404e8bb12b25ca8ea97052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:56.765928 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5980746-d6b2-44d7-ad62-203d48f68990 service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:56.766209 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5980746-d6b2-44d7-ad62-203d48f68990 service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:56.766479 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5980746-d6b2-44d7-ad62-203d48f68990 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:56.778165 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5980746-d6b2-44d7-ad62-203d48f68990 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:56.780639 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1259/2535] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 10:03:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 10:03:56.780759 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5980746-d6b2-44d7-ad62-203d48f68990 service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:56.789582 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1277/2536] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 10:03:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:03:56.884577 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:57.427985 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff377e9-b318-4c86-bf0d-883a71f02aac service neutron] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:57.429769 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-aff377e9-b318-4c86-bf0d-883a71f02aac service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:57.478338 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5980746-d6b2-44d7-ad62-203d48f68990 service neutron] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:57.479947 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-e5980746-d6b2-44d7-ad62-203d48f68990 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:57.495674 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5980746-d6b2-44d7-ad62-203d48f68990 service neutron] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:57.504347 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba023c2-9719-42bd-8eab-ce76a04b86af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:57.504347 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba023c2-9719-42bd-8eab-ce76a04b86af admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:57.504347 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba023c2-9719-42bd-8eab-ce76a04b86af admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:57.504347 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba023c2-9719-42bd-8eab-ce76a04b86af admin admin] PATH_INFO: `/v3/projects/37bf221d7438497e9ea138f0c18d1e35` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:57.504701 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba023c2-9719-42bd-8eab-ce76a04b86af admin admin] RBAC: Authorizing `identity:delete_project(project_id=37bf221d7438497e9ea138f0c18d1e35)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:57.506450 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba023c2-9719-42bd-8eab-ce76a04b86af admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:57.511734 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff377e9-b318-4c86-bf0d-883a71f02aac service neutron] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:57.518187 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c154e12d-eb89-4a8a-b735-96423cf560e7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80cada22039d4f11935412ee578acd05', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:57.520247 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c154e12d-eb89-4a8a-b735-96423cf560e7 service placement] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:57.520247 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c154e12d-eb89-4a8a-b735-96423cf560e7 service placement] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:57.520426 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c154e12d-eb89-4a8a-b735-96423cf560e7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:57.522135 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-c154e12d-eb89-4a8a-b735-96423cf560e7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:57.536721 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c154e12d-eb89-4a8a-b735-96423cf560e7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:57.538347 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c154e12d-eb89-4a8a-b735-96423cf560e7 service placement] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:57.548429 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1260/2537] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 10:03:57 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:03:57.549729 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c154e12d-eb89-4a8a-b735-96423cf560e7 service placement] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:57.551027 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-c154e12d-eb89-4a8a-b735-96423cf560e7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:57.563544 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c154e12d-eb89-4a8a-b735-96423cf560e7 service placement] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:57.568675 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be52b13-657d-4922-bd69-c020b883309d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:57.569297 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be52b13-657d-4922-bd69-c020b883309d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:57.569576 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be52b13-657d-4922-bd69-c020b883309d admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:57.569844 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be52b13-657d-4922-bd69-c020b883309d admin admin] PATH_INFO: `/v3/projects/caf4ada5edb94caf8f918b16599609ae` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:57.570973 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be52b13-657d-4922-bd69-c020b883309d admin admin] RBAC: Authorizing `identity:delete_project(project_id=caf4ada5edb94caf8f918b16599609ae)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:57.572893 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be52b13-657d-4922-bd69-c020b883309d admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:57.635031 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-9ba023c2-9719-42bd-8eab-ce76a04b86af admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '37bf221d7438497e9ea138f0c18d1e35', 'request_id': 'req-9ba023c2-9719-42bd-8eab-ce76a04b86af'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:57.642079 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-9ba023c2-9719-42bd-8eab-ce76a04b86af admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '37bf221d7438497e9ea138f0c18d1e35', 'request_id': 'req-9ba023c2-9719-42bd-8eab-ce76a04b86af'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:57.647964 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-9ba023c2-9719-42bd-8eab-ce76a04b86af admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '37bf221d7438497e9ea138f0c18d1e35', 'request_id': 'req-9ba023c2-9719-42bd-8eab-ce76a04b86af'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:57.706674 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:57.708137 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-3be52b13-657d-4922-bd69-c020b883309d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'caf4ada5edb94caf8f918b16599609ae', 'request_id': 'req-3be52b13-657d-4922-bd69-c020b883309d'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:57.714330 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-3be52b13-657d-4922-bd69-c020b883309d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'caf4ada5edb94caf8f918b16599609ae', 'request_id': 'req-3be52b13-657d-4922-bd69-c020b883309d'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:57.719901 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-3be52b13-657d-4922-bd69-c020b883309d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'caf4ada5edb94caf8f918b16599609ae', 'request_id': 'req-3be52b13-657d-4922-bd69-c020b883309d'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:57.728906 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-9ba023c2-9719-42bd-8eab-ce76a04b86af admin admin] The token cache is being invalidate because project 37bf221d7438497e9ea138f0c18d1e35 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81602) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:03:57.729204 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-9ba023c2-9719-42bd-8eab-ce76a04b86af admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:57.729867 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-9ba023c2-9719-42bd-8eab-ce76a04b86af admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:57.731376 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1278/2538] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 24 10:03:57 2022] DELETE /identity/v3/projects/37bf221d7438497e9ea138f0c18d1e35 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:03:57.735109 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba023c2-9719-42bd-8eab-ce76a04b86af admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:57.736463 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-9ba023c2-9719-42bd-8eab-ce76a04b86af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:57.764003 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:57.801059 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba023c2-9719-42bd-8eab-ce76a04b86af admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:57.822280 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-3be52b13-657d-4922-bd69-c020b883309d admin admin] The token cache is being invalidate because project caf4ada5edb94caf8f918b16599609ae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:03:57.822573 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-3be52b13-657d-4922-bd69-c020b883309d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:57.828326 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-3be52b13-657d-4922-bd69-c020b883309d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:57.828326 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1261/2539] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 24 10:03:57 2022] DELETE /identity/v3/projects/caf4ada5edb94caf8f918b16599609ae => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:03:57.828749 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be52b13-657d-4922-bd69-c020b883309d admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:57.830980 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-3be52b13-657d-4922-bd69-c020b883309d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:57.858539 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9510119-e8c7-4b97-a90c-fc3ac7aa7356 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:57.859185 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9510119-e8c7-4b97-a90c-fc3ac7aa7356 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:57.859469 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9510119-e8c7-4b97-a90c-fc3ac7aa7356 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:57.859813 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9510119-e8c7-4b97-a90c-fc3ac7aa7356 admin admin] PATH_INFO: `/v3/users/36546304609647538d464396ed4e8496` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:57.860999 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9510119-e8c7-4b97-a90c-fc3ac7aa7356 admin admin] RBAC: Authorizing `identity:delete_user(user_id=36546304609647538d464396ed4e8496)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:57.863044 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9510119-e8c7-4b97-a90c-fc3ac7aa7356 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:57.894386 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be52b13-657d-4922-bd69-c020b883309d admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:57.899159 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a10b7e1-c279-4b0a-b9d1-3579fbca5544 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:57.899834 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a10b7e1-c279-4b0a-b9d1-3579fbca5544 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:57.900116 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a10b7e1-c279-4b0a-b9d1-3579fbca5544 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:57.900391 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a10b7e1-c279-4b0a-b9d1-3579fbca5544 admin admin] PATH_INFO: `/v3/users/9b0fcdf55c80405da239f221af9805c6` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:57.901525 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a10b7e1-c279-4b0a-b9d1-3579fbca5544 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9b0fcdf55c80405da239f221af9805c6)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:57.903445 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a10b7e1-c279-4b0a-b9d1-3579fbca5544 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:58.075380 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-b9510119-e8c7-4b97-a90c-fc3ac7aa7356 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '36546304609647538d464396ed4e8496'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:58.075990 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-b9510119-e8c7-4b97-a90c-fc3ac7aa7356 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '36546304609647538d464396ed4e8496'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:58.105267 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-1a10b7e1-c279-4b0a-b9d1-3579fbca5544 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9b0fcdf55c80405da239f221af9805c6'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:58.106008 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-1a10b7e1-c279-4b0a-b9d1-3579fbca5544 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9b0fcdf55c80405da239f221af9805c6'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:58.122844 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-1a10b7e1-c279-4b0a-b9d1-3579fbca5544 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9b0fcdf55c80405da239f221af9805c6'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:58.126678 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-1a10b7e1-c279-4b0a-b9d1-3579fbca5544 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9b0fcdf55c80405da239f221af9805c6'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:58.135621 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-b9510119-e8c7-4b97-a90c-fc3ac7aa7356 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '36546304609647538d464396ed4e8496'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:58.136296 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-b9510119-e8c7-4b97-a90c-fc3ac7aa7356 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '36546304609647538d464396ed4e8496'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:58.145086 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-b9510119-e8c7-4b97-a90c-fc3ac7aa7356 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '36546304609647538d464396ed4e8496'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:58.164749 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1279/2540] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 24 10:03:57 2022] DELETE /identity/v3/users/36546304609647538d464396ed4e8496 => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 10:03:58.226729 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-1a10b7e1-c279-4b0a-b9d1-3579fbca5544 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9b0fcdf55c80405da239f221af9805c6'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:58.233435 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1262/2541] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 24 10:03:57 2022] DELETE /identity/v3/users/9b0fcdf55c80405da239f221af9805c6 => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 10:03:58.388590 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9510119-e8c7-4b97-a90c-fc3ac7aa7356 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:58.390216 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-b9510119-e8c7-4b97-a90c-fc3ac7aa7356 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:58.461574 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9510119-e8c7-4b97-a90c-fc3ac7aa7356 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:58.480128 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:58.539134 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eca3bca-219d-45e0-8e90-14fcbf0898f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e5260e798404e8bb12b25ca8ea97052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:58.539895 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eca3bca-219d-45e0-8e90-14fcbf0898f4 service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:58.540605 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eca3bca-219d-45e0-8e90-14fcbf0898f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:58.541107 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eca3bca-219d-45e0-8e90-14fcbf0898f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:58.543327 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-5eca3bca-219d-45e0-8e90-14fcbf0898f4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:58.650678 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eca3bca-219d-45e0-8e90-14fcbf0898f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:58.652376 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eca3bca-219d-45e0-8e90-14fcbf0898f4 service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:58.662932 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1280/2542] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 10:03:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 275 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:03:58.717660 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a10b7e1-c279-4b0a-b9d1-3579fbca5544 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:58.719105 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-1a10b7e1-c279-4b0a-b9d1-3579fbca5544 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:58.738726 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a10b7e1-c279-4b0a-b9d1-3579fbca5544 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:58.743817 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0034cb-55d4-4ef7-a80b-801d5685b4ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:58.745413 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0034cb-55d4-4ef7-a80b-801d5685b4ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:58.745935 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0034cb-55d4-4ef7-a80b-801d5685b4ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:58.746223 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0034cb-55d4-4ef7-a80b-801d5685b4ba admin admin] PATH_INFO: `/v3/projects/b3808bba24664822a72296a290f5ceef` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:58.747950 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0034cb-55d4-4ef7-a80b-801d5685b4ba admin admin] RBAC: Authorizing `identity:delete_project(project_id=b3808bba24664822a72296a290f5ceef)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:58.749840 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0034cb-55d4-4ef7-a80b-801d5685b4ba admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:58.806015 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eca3bca-219d-45e0-8e90-14fcbf0898f4 service neutron] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:58.807761 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-5eca3bca-219d-45e0-8e90-14fcbf0898f4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:58.820122 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eca3bca-219d-45e0-8e90-14fcbf0898f4 service neutron] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:58.825872 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5412a9-6a20-4bf1-9bce-4458034194ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:58.826638 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5412a9-6a20-4bf1-9bce-4458034194ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:58.827014 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5412a9-6a20-4bf1-9bce-4458034194ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:58.827402 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5412a9-6a20-4bf1-9bce-4458034194ee admin admin] PATH_INFO: `/v3/projects/b10250b5d4c647c1a2c42544aeb0b2ef` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:58.828755 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5412a9-6a20-4bf1-9bce-4458034194ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=b10250b5d4c647c1a2c42544aeb0b2ef)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:58.832373 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5412a9-6a20-4bf1-9bce-4458034194ee admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:58.895709 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-ed0034cb-55d4-4ef7-a80b-801d5685b4ba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b3808bba24664822a72296a290f5ceef', 'request_id': 'req-ed0034cb-55d4-4ef7-a80b-801d5685b4ba'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:58.896822 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:58.902548 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-ed0034cb-55d4-4ef7-a80b-801d5685b4ba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b3808bba24664822a72296a290f5ceef', 'request_id': 'req-ed0034cb-55d4-4ef7-a80b-801d5685b4ba'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:58.908791 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-ed0034cb-55d4-4ef7-a80b-801d5685b4ba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b3808bba24664822a72296a290f5ceef', 'request_id': 'req-ed0034cb-55d4-4ef7-a80b-801d5685b4ba'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:58.947948 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-ba5412a9-6a20-4bf1-9bce-4458034194ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b10250b5d4c647c1a2c42544aeb0b2ef', 'request_id': 'req-ba5412a9-6a20-4bf1-9bce-4458034194ee'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:58.959443 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-ba5412a9-6a20-4bf1-9bce-4458034194ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b10250b5d4c647c1a2c42544aeb0b2ef', 'request_id': 'req-ba5412a9-6a20-4bf1-9bce-4458034194ee'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:58.965601 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-ba5412a9-6a20-4bf1-9bce-4458034194ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b10250b5d4c647c1a2c42544aeb0b2ef', 'request_id': 'req-ba5412a9-6a20-4bf1-9bce-4458034194ee'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:59.015214 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-ed0034cb-55d4-4ef7-a80b-801d5685b4ba admin admin] The token cache is being invalidate because project b3808bba24664822a72296a290f5ceef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:03:59.015508 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-ed0034cb-55d4-4ef7-a80b-801d5685b4ba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:59.016160 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-ed0034cb-55d4-4ef7-a80b-801d5685b4ba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:59.018105 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1263/2543] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 24 10:03:58 2022] DELETE /identity/v3/projects/b3808bba24664822a72296a290f5ceef => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:03:59.021977 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0034cb-55d4-4ef7-a80b-801d5685b4ba admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:59.023295 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-ed0034cb-55d4-4ef7-a80b-801d5685b4ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:59.098408 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0034cb-55d4-4ef7-a80b-801d5685b4ba admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:59.098609 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-ba5412a9-6a20-4bf1-9bce-4458034194ee admin admin] The token cache is being invalidate because project b10250b5d4c647c1a2c42544aeb0b2ef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81602) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:03:59.098609 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-ba5412a9-6a20-4bf1-9bce-4458034194ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:59.099117 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-ba5412a9-6a20-4bf1-9bce-4458034194ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:59.101159 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1281/2544] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 24 10:03:58 2022] DELETE /identity/v3/projects/b10250b5d4c647c1a2c42544aeb0b2ef => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 10:03:59.106178 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd031fc-37fd-4aae-9ea6-98d0532d68c3 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:59.106473 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd031fc-37fd-4aae-9ea6-98d0532d68c3 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:59.106730 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd031fc-37fd-4aae-9ea6-98d0532d68c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:59.128888 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-6fd031fc-37fd-4aae-9ea6-98d0532d68c3 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:03:59.130138 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-6fd031fc-37fd-4aae-9ea6-98d0532d68c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:59.159905 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef6ce43-ea1b-4d98-8442-0b6036fe9ba2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:59.160913 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef6ce43-ea1b-4d98-8442-0b6036fe9ba2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:59.161372 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef6ce43-ea1b-4d98-8442-0b6036fe9ba2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:59.161744 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef6ce43-ea1b-4d98-8442-0b6036fe9ba2 admin admin] PATH_INFO: `/v3/users/6832fdc882884cd8bc9071b44a70efbc` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:59.163059 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef6ce43-ea1b-4d98-8442-0b6036fe9ba2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6832fdc882884cd8bc9071b44a70efbc)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:59.165225 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef6ce43-ea1b-4d98-8442-0b6036fe9ba2 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:59.195865 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1282/2545] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 10:03:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:03:59.204696 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15b78f9-0ae6-410b-837b-78095ae5ef50 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:59.204988 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15b78f9-0ae6-410b-837b-78095ae5ef50 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:59.205794 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15b78f9-0ae6-410b-837b-78095ae5ef50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:59.225670 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-f15b78f9-0ae6-410b-837b-78095ae5ef50 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:03:59.228176 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:59.228940 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-f15b78f9-0ae6-410b-837b-78095ae5ef50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:59.237679 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1283/2546] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 10:03:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:03:59.241722 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15b78f9-0ae6-410b-837b-78095ae5ef50 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:59.252032 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15b78f9-0ae6-410b-837b-78095ae5ef50 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:59.260802 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02dcb1d-8939-4f74-bfff-68f45222a837 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:59.261896 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02dcb1d-8939-4f74-bfff-68f45222a837 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:59.264898 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02dcb1d-8939-4f74-bfff-68f45222a837 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:59.264898 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02dcb1d-8939-4f74-bfff-68f45222a837 admin admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:59.264898 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02dcb1d-8939-4f74-bfff-68f45222a837 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:03:59.264898 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02dcb1d-8939-4f74-bfff-68f45222a837 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:59.265519 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02dcb1d-8939-4f74-bfff-68f45222a837 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:59.281310 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1284/2547] 10.222.0.16 () {58 vars in 1153 bytes} [Wed Aug 24 10:03:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 10:03:59.282601 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02dcb1d-8939-4f74-bfff-68f45222a837 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:59.293132 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02dcb1d-8939-4f74-bfff-68f45222a837 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:59.297970 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950db3af-8e21-44eb-8620-e8d2d8be8cbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:59.298577 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950db3af-8e21-44eb-8620-e8d2d8be8cbd admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:59.298914 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950db3af-8e21-44eb-8620-e8d2d8be8cbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:59.303760 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950db3af-8e21-44eb-8620-e8d2d8be8cbd admin admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:59.303760 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950db3af-8e21-44eb-8620-e8d2d8be8cbd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:59.303760 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950db3af-8e21-44eb-8620-e8d2d8be8cbd admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:59.343858 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-fef6ce43-ea1b-4d98-8442-0b6036fe9ba2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6832fdc882884cd8bc9071b44a70efbc'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:59.344537 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-fef6ce43-ea1b-4d98-8442-0b6036fe9ba2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6832fdc882884cd8bc9071b44a70efbc'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:59.357372 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-fef6ce43-ea1b-4d98-8442-0b6036fe9ba2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6832fdc882884cd8bc9071b44a70efbc'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:59.364067 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-fef6ce43-ea1b-4d98-8442-0b6036fe9ba2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6832fdc882884cd8bc9071b44a70efbc'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:59.433778 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1285/2548] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 24 10:03:59 2022] POST /identity/v3/projects => generated 365 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 10:03:59.434767 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950db3af-8e21-44eb-8620-e8d2d8be8cbd admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:59.436527 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-950db3af-8e21-44eb-8620-e8d2d8be8cbd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:59.436699 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-fef6ce43-ea1b-4d98-8442-0b6036fe9ba2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6832fdc882884cd8bc9071b44a70efbc'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:59.443425 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1264/2549] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 24 10:03:59 2022] DELETE /identity/v3/users/6832fdc882884cd8bc9071b44a70efbc => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 10:03:59.444241 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef6ce43-ea1b-4d98-8442-0b6036fe9ba2 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:59.445552 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-fef6ce43-ea1b-4d98-8442-0b6036fe9ba2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:59.511083 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950db3af-8e21-44eb-8620-e8d2d8be8cbd admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:59.514466 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef6ce43-ea1b-4d98-8442-0b6036fe9ba2 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:59.516080 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbf1bd9-ff54-41a8-9a22-e7842a6b2c90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:59.516878 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbf1bd9-ff54-41a8-9a22-e7842a6b2c90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:59.517017 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbf1bd9-ff54-41a8-9a22-e7842a6b2c90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:59.517270 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbf1bd9-ff54-41a8-9a22-e7842a6b2c90 admin admin] PATH_INFO: `/v3/users/bfc3f904a4fc4abc91e8df52ec816a0a` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:59.519006 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbf1bd9-ff54-41a8-9a22-e7842a6b2c90 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bfc3f904a4fc4abc91e8df52ec816a0a)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:59.520519 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739bc85b-6732-492e-8a0a-7cde7283fc86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:59.521111 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbf1bd9-ff54-41a8-9a22-e7842a6b2c90 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:59.521215 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739bc85b-6732-492e-8a0a-7cde7283fc86 admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:59.521774 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739bc85b-6732-492e-8a0a-7cde7283fc86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:59.521774 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739bc85b-6732-492e-8a0a-7cde7283fc86 admin admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:59.522438 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739bc85b-6732-492e-8a0a-7cde7283fc86 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:59.524397 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739bc85b-6732-492e-8a0a-7cde7283fc86 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:59.623618 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1265/2550] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 24 10:03:59 2022] POST /identity/v3/users => generated 283 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:03:59.623618 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739bc85b-6732-492e-8a0a-7cde7283fc86 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:59.628681 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739bc85b-6732-492e-8a0a-7cde7283fc86 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:59.633278 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e860b7a-1ad2-44be-825f-5c6f46a63275 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:59.633920 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e860b7a-1ad2-44be-825f-5c6f46a63275 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:59.634189 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e860b7a-1ad2-44be-825f-5c6f46a63275 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:59.634511 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e860b7a-1ad2-44be-825f-5c6f46a63275 admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:59.635164 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e860b7a-1ad2-44be-825f-5c6f46a63275 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:59.636778 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e860b7a-1ad2-44be-825f-5c6f46a63275 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:59.652920 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1266/2551] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 24 10:03:59 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 10:03:59.658774 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e860b7a-1ad2-44be-825f-5c6f46a63275 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:59.669911 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e860b7a-1ad2-44be-825f-5c6f46a63275 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:59.675169 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d5eaef-49a2-4ec7-ad03-69a5564b2472 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:59.676052 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d5eaef-49a2-4ec7-ad03-69a5564b2472 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:59.676775 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d5eaef-49a2-4ec7-ad03-69a5564b2472 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:59.677062 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d5eaef-49a2-4ec7-ad03-69a5564b2472 admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:59.677764 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d5eaef-49a2-4ec7-ad03-69a5564b2472 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:59.679437 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d5eaef-49a2-4ec7-ad03-69a5564b2472 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:59.701345 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1267/2552] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 24 10:03:59 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:03:59.702402 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d5eaef-49a2-4ec7-ad03-69a5564b2472 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:59.714094 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d5eaef-49a2-4ec7-ad03-69a5564b2472 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:59.726138 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a68b75-b096-4810-9502-b59aa1c5a37e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:59.726945 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a68b75-b096-4810-9502-b59aa1c5a37e admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:59.727261 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a68b75-b096-4810-9502-b59aa1c5a37e admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:59.727664 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a68b75-b096-4810-9502-b59aa1c5a37e admin admin] PATH_INFO: `/v3/projects/f5e76a8ec44d48a4a998705a74b36f76/users/7c280ab9e891443a91c6861a89890e9f/roles/136cdf78e00a426997dd4e37faff489b` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:59.748083 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a68b75-b096-4810-9502-b59aa1c5a37e admin admin] RBAC: Authorizing `identity:create_grant(project_id=f5e76a8ec44d48a4a998705a74b36f76, user_id=7c280ab9e891443a91c6861a89890e9f, role_id=136cdf78e00a426997dd4e37faff489b)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:59.751068 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a68b75-b096-4810-9502-b59aa1c5a37e admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:59.753724 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-1bbf1bd9-ff54-41a8-9a22-e7842a6b2c90 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bfc3f904a4fc4abc91e8df52ec816a0a'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:59.754378 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-1bbf1bd9-ff54-41a8-9a22-e7842a6b2c90 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bfc3f904a4fc4abc91e8df52ec816a0a'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:59.861142 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1268/2553] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 24 10:03:59 2022] PUT /identity/v3/projects/f5e76a8ec44d48a4a998705a74b36f76/users/7c280ab9e891443a91c6861a89890e9f/roles/136cdf78e00a426997dd4e37faff489b => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:03:59.871434 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebf0312-5fa8-45e2-a58a-1a9a8ac15efa None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:59.871956 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebf0312-5fa8-45e2-a58a-1a9a8ac15efa None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:59.872554 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebf0312-5fa8-45e2-a58a-1a9a8ac15efa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:59.872810 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-1bbf1bd9-ff54-41a8-9a22-e7842a6b2c90 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bfc3f904a4fc4abc91e8df52ec816a0a'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:59.873771 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-1bbf1bd9-ff54-41a8-9a22-e7842a6b2c90 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bfc3f904a4fc4abc91e8df52ec816a0a'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:59.882532 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-1bbf1bd9-ff54-41a8-9a22-e7842a6b2c90 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bfc3f904a4fc4abc91e8df52ec816a0a'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:59.903877 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1286/2554] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 24 10:03:59 2022] DELETE /identity/v3/users/bfc3f904a4fc4abc91e8df52ec816a0a => generated 0 bytes in 470 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:03:59.904810 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbf1bd9-ff54-41a8-9a22-e7842a6b2c90 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:59.907206 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-eebf0312-5fa8-45e2-a58a-1a9a8ac15efa None None] MFA Rules not processed for user `7c280ab9e891443a91c6861a89890e9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:03:59.908578 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-eebf0312-5fa8-45e2-a58a-1a9a8ac15efa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:59.912724 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-1bbf1bd9-ff54-41a8-9a22-e7842a6b2c90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:59.946187 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:59.995317 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbf1bd9-ff54-41a8-9a22-e7842a6b2c90 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:00.004392 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1269/2555] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 10:03:59 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:04:00.004594 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71fe231-7859-4790-abee-a56f3264599a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:00.004594 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71fe231-7859-4790-abee-a56f3264599a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:00.004594 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71fe231-7859-4790-abee-a56f3264599a admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:00.004594 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71fe231-7859-4790-abee-a56f3264599a admin admin] PATH_INFO: `/v3/projects/37fbb021aeb9463f9d85f1d04f3e1e4a` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:00.004935 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71fe231-7859-4790-abee-a56f3264599a admin admin] RBAC: Authorizing `identity:delete_project(project_id=37fbb021aeb9463f9d85f1d04f3e1e4a)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:00.006625 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71fe231-7859-4790-abee-a56f3264599a admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:00.006882 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebf0312-5fa8-45e2-a58a-1a9a8ac15efa None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:00.012694 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-eebf0312-5fa8-45e2-a58a-1a9a8ac15efa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:00.031992 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebf0312-5fa8-45e2-a58a-1a9a8ac15efa None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:00.037523 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62d3d5c-b572-495b-88e5-b9bed86e82b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:00.038150 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62d3d5c-b572-495b-88e5-b9bed86e82b4 admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:00.038439 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62d3d5c-b572-495b-88e5-b9bed86e82b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:00.038715 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62d3d5c-b572-495b-88e5-b9bed86e82b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:00.039987 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62d3d5c-b572-495b-88e5-b9bed86e82b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:00.041922 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62d3d5c-b572-495b-88e5-b9bed86e82b4 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:00.106354 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-a71fe231-7859-4790-abee-a56f3264599a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '37fbb021aeb9463f9d85f1d04f3e1e4a', 'request_id': 'req-a71fe231-7859-4790-abee-a56f3264599a'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:00.113241 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-a71fe231-7859-4790-abee-a56f3264599a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '37fbb021aeb9463f9d85f1d04f3e1e4a', 'request_id': 'req-a71fe231-7859-4790-abee-a56f3264599a'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:00.118831 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-a71fe231-7859-4790-abee-a56f3264599a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '37fbb021aeb9463f9d85f1d04f3e1e4a', 'request_id': 'req-a71fe231-7859-4790-abee-a56f3264599a'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:00.195067 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-a71fe231-7859-4790-abee-a56f3264599a admin admin] The token cache is being invalidate because project 37fbb021aeb9463f9d85f1d04f3e1e4a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81602) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:04:00.195444 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-a71fe231-7859-4790-abee-a56f3264599a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:00.196211 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-a71fe231-7859-4790-abee-a56f3264599a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:00.196697 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1270/2556] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 24 10:04:00 2022] POST /identity/v3/projects => generated 369 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:04:00.197900 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1287/2557] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 24 10:03:59 2022] DELETE /identity/v3/projects/37fbb021aeb9463f9d85f1d04f3e1e4a => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:04:00.200821 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62d3d5c-b572-495b-88e5-b9bed86e82b4 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:00.203287 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71fe231-7859-4790-abee-a56f3264599a admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:00.203514 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-f62d3d5c-b572-495b-88e5-b9bed86e82b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:00.204858 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-a71fe231-7859-4790-abee-a56f3264599a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:00.250954 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:04:00.269687 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62d3d5c-b572-495b-88e5-b9bed86e82b4 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:00.275020 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28509720-7390-4659-9430-fcec5ef19243 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:00.275307 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71fe231-7859-4790-abee-a56f3264599a admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:00.275682 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28509720-7390-4659-9430-fcec5ef19243 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:00.275965 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28509720-7390-4659-9430-fcec5ef19243 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:00.276238 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28509720-7390-4659-9430-fcec5ef19243 admin admin] PATH_INFO: `/v3/projects/0b3e1d0646174986919db290c6da2579` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:00.277352 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28509720-7390-4659-9430-fcec5ef19243 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b3e1d0646174986919db290c6da2579)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:00.279292 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28509720-7390-4659-9430-fcec5ef19243 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:00.281058 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5cf1d6-646c-459b-ba2f-af592c2fbb50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:00.281779 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5cf1d6-646c-459b-ba2f-af592c2fbb50 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:00.282056 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5cf1d6-646c-459b-ba2f-af592c2fbb50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:00.282326 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5cf1d6-646c-459b-ba2f-af592c2fbb50 admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:00.282946 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5cf1d6-646c-459b-ba2f-af592c2fbb50 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:00.285394 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5cf1d6-646c-459b-ba2f-af592c2fbb50 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:00.350003 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1288/2558] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 24 10:04:00 2022] POST /identity/v3/users => generated 291 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:04:00.350003 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25960094-8112-40f8-80e3-3d610d082184 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:00.350003 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25960094-8112-40f8-80e3-3d610d082184 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:00.350003 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25960094-8112-40f8-80e3-3d610d082184 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:00.368849 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-25960094-8112-40f8-80e3-3d610d082184 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:04:00.370133 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-25960094-8112-40f8-80e3-3d610d082184 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:00.381575 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-28509720-7390-4659-9430-fcec5ef19243 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b3e1d0646174986919db290c6da2579', 'request_id': 'req-28509720-7390-4659-9430-fcec5ef19243'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:00.387770 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-28509720-7390-4659-9430-fcec5ef19243 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b3e1d0646174986919db290c6da2579', 'request_id': 'req-28509720-7390-4659-9430-fcec5ef19243'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:00.393849 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-28509720-7390-4659-9430-fcec5ef19243 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b3e1d0646174986919db290c6da2579', 'request_id': 'req-28509720-7390-4659-9430-fcec5ef19243'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:00.430302 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1289/2559] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 10:04:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:04:00.431209 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25960094-8112-40f8-80e3-3d610d082184 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:00.440992 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25960094-8112-40f8-80e3-3d610d082184 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:00.445841 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed01cb4d-f8ec-4885-97dd-506deedd6379 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:00.446444 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed01cb4d-f8ec-4885-97dd-506deedd6379 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:00.446726 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed01cb4d-f8ec-4885-97dd-506deedd6379 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:00.447001 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed01cb4d-f8ec-4885-97dd-506deedd6379 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:00.447676 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed01cb4d-f8ec-4885-97dd-506deedd6379 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:00.449310 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed01cb4d-f8ec-4885-97dd-506deedd6379 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:00.459991 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1290/2560] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 24 10:04:00 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:04:00.461148 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc8d047-23fc-45ca-ae89-82e97f2aa973 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:00.461427 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc8d047-23fc-45ca-ae89-82e97f2aa973 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:00.461698 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc8d047-23fc-45ca-ae89-82e97f2aa973 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:00.477733 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-28509720-7390-4659-9430-fcec5ef19243 admin admin] The token cache is being invalidate because project 0b3e1d0646174986919db290c6da2579 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:04:00.478036 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-28509720-7390-4659-9430-fcec5ef19243 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:00.478710 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-28509720-7390-4659-9430-fcec5ef19243 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:00.480309 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1271/2561] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 24 10:04:00 2022] DELETE /identity/v3/projects/0b3e1d0646174986919db290c6da2579 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:04:00.481530 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-3bc8d047-23fc-45ca-ae89-82e97f2aa973 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:04:00.481637 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28509720-7390-4659-9430-fcec5ef19243 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:00.482116 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-3bc8d047-23fc-45ca-ae89-82e97f2aa973 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:00.482467 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-28509720-7390-4659-9430-fcec5ef19243 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:00.500346 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1291/2562] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 10:04:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:04:00.504420 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28509720-7390-4659-9430-fcec5ef19243 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:00.505657 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc8d047-23fc-45ca-ae89-82e97f2aa973 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:00.509777 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be39d6a-deda-42a6-a451-c0565a227cd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:00.512124 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be39d6a-deda-42a6-a451-c0565a227cd0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:00.512124 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be39d6a-deda-42a6-a451-c0565a227cd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:00.512124 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be39d6a-deda-42a6-a451-c0565a227cd0 admin admin] PATH_INFO: `/v3/projects/0074f864f9884477803dd6b1ce858b00/users/7ff7d3b8e0c346ba8257e39d45998ba9/roles/15f634d4a2d247ddab90c458e5e1c141` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:00.516667 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc8d047-23fc-45ca-ae89-82e97f2aa973 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:00.523548 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd8349b-ba29-43d8-ac65-c85abff759c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:00.524233 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd8349b-ba29-43d8-ac65-c85abff759c4 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:00.524542 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd8349b-ba29-43d8-ac65-c85abff759c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:00.524813 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd8349b-ba29-43d8-ac65-c85abff759c4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:00.525448 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd8349b-ba29-43d8-ac65-c85abff759c4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:04:00.525780 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd8349b-ba29-43d8-ac65-c85abff759c4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:00.527444 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd8349b-ba29-43d8-ac65-c85abff759c4 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:00.530695 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be39d6a-deda-42a6-a451-c0565a227cd0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0074f864f9884477803dd6b1ce858b00, user_id=7ff7d3b8e0c346ba8257e39d45998ba9, role_id=15f634d4a2d247ddab90c458e5e1c141)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:00.532524 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:04:00.536423 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be39d6a-deda-42a6-a451-c0565a227cd0 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:00.543630 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1292/2563] 10.222.0.16 () {58 vars in 1153 bytes} [Wed Aug 24 10:04:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 10:04:00.547207 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd8349b-ba29-43d8-ac65-c85abff759c4 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:00.557869 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd8349b-ba29-43d8-ac65-c85abff759c4 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:00.563143 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50f10f6-fdb9-417e-bf2f-4488029b2d89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:00.564127 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50f10f6-fdb9-417e-bf2f-4488029b2d89 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:00.564969 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50f10f6-fdb9-417e-bf2f-4488029b2d89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:00.565264 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50f10f6-fdb9-417e-bf2f-4488029b2d89 admin admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:00.565887 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50f10f6-fdb9-417e-bf2f-4488029b2d89 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:00.567964 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50f10f6-fdb9-417e-bf2f-4488029b2d89 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:00.604758 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1272/2564] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 24 10:04:00 2022] PUT /identity/v3/projects/0074f864f9884477803dd6b1ce858b00/users/7ff7d3b8e0c346ba8257e39d45998ba9/roles/15f634d4a2d247ddab90c458e5e1c141 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:04:00.609270 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be39d6a-deda-42a6-a451-c0565a227cd0 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:00.664269 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1293/2565] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 24 10:04:00 2022] POST /identity/v3/projects => generated 399 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 10:04:00.673350 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50f10f6-fdb9-417e-bf2f-4488029b2d89 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:00.684767 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be39d6a-deda-42a6-a451-c0565a227cd0 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:00.698256 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:04:00.719081 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:04:00.740130 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50f10f6-fdb9-417e-bf2f-4488029b2d89 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:00.744635 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ece6b48-b5e1-4301-8e9d-dde55b5f7bc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:00.745157 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6aeb8f-85a6-49ad-a05f-a8feb91728ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:00.745290 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ece6b48-b5e1-4301-8e9d-dde55b5f7bc2 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:00.745388 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6aeb8f-85a6-49ad-a05f-a8feb91728ec admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:00.745729 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ece6b48-b5e1-4301-8e9d-dde55b5f7bc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:00.745840 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ece6b48-b5e1-4301-8e9d-dde55b5f7bc2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:00.746412 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ece6b48-b5e1-4301-8e9d-dde55b5f7bc2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:00.746597 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6aeb8f-85a6-49ad-a05f-a8feb91728ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:00.746957 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6aeb8f-85a6-49ad-a05f-a8feb91728ec admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:00.747596 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6aeb8f-85a6-49ad-a05f-a8feb91728ec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:00.748076 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ece6b48-b5e1-4301-8e9d-dde55b5f7bc2 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:00.749518 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6aeb8f-85a6-49ad-a05f-a8feb91728ec admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:00.762474 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1273/2566] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 24 10:04:00 2022] GET /identity/v3/roles => generated 1420 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:04:00.763510 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ece6b48-b5e1-4301-8e9d-dde55b5f7bc2 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:00.773238 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ece6b48-b5e1-4301-8e9d-dde55b5f7bc2 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:00.778354 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024df1fa-fc93-4aa2-a26b-514965ad30b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:00.779143 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024df1fa-fc93-4aa2-a26b-514965ad30b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:00.779292 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024df1fa-fc93-4aa2-a26b-514965ad30b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:00.779542 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024df1fa-fc93-4aa2-a26b-514965ad30b6 admin admin] PATH_INFO: `/v3/projects/0074f864f9884477803dd6b1ce858b00/users/7ff7d3b8e0c346ba8257e39d45998ba9/roles/15f634d4a2d247ddab90c458e5e1c141` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:00.781506 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024df1fa-fc93-4aa2-a26b-514965ad30b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0074f864f9884477803dd6b1ce858b00, user_id=7ff7d3b8e0c346ba8257e39d45998ba9, role_id=15f634d4a2d247ddab90c458e5e1c141)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:00.784212 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024df1fa-fc93-4aa2-a26b-514965ad30b6 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:00.794445 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1274/2567] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 24 10:04:00 2022] PUT /identity/v3/projects/0074f864f9884477803dd6b1ce858b00/users/7ff7d3b8e0c346ba8257e39d45998ba9/roles/15f634d4a2d247ddab90c458e5e1c141 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:04:00.805489 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3c0e5b-4895-4d47-9c1d-a2dc27464804 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:00.805780 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3c0e5b-4895-4d47-9c1d-a2dc27464804 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:00.806064 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3c0e5b-4895-4d47-9c1d-a2dc27464804 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:00.835098 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1294/2568] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 24 10:04:00 2022] POST /identity/v3/users => generated 300 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 10:04:00.835289 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-5c3c0e5b-4895-4d47-9c1d-a2dc27464804 None None] MFA Rules not processed for user `7ff7d3b8e0c346ba8257e39d45998ba9`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:04:00.836434 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-5c3c0e5b-4895-4d47-9c1d-a2dc27464804 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:00.839890 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6aeb8f-85a6-49ad-a05f-a8feb91728ec admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:00.852961 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6aeb8f-85a6-49ad-a05f-a8feb91728ec admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:00.906381 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1275/2569] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 10:04:00 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:04:00.909457 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efdb244-ec5a-47f8-bdd2-48bcbe2afc1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:00.910088 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efdb244-ec5a-47f8-bdd2-48bcbe2afc1a admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:00.910371 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efdb244-ec5a-47f8-bdd2-48bcbe2afc1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:00.910651 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efdb244-ec5a-47f8-bdd2-48bcbe2afc1a admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:00.911311 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efdb244-ec5a-47f8-bdd2-48bcbe2afc1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:00.913089 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efdb244-ec5a-47f8-bdd2-48bcbe2afc1a admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:00.917743 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3c0e5b-4895-4d47-9c1d-a2dc27464804 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:00.919607 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-5c3c0e5b-4895-4d47-9c1d-a2dc27464804 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:00.923202 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1295/2570] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 24 10:04:00 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:04:00.927765 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efdb244-ec5a-47f8-bdd2-48bcbe2afc1a admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:00.938970 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efdb244-ec5a-47f8-bdd2-48bcbe2afc1a admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:00.944350 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1fb68b-d7c4-4afd-89a6-4b945c26acbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:00.944943 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1fb68b-d7c4-4afd-89a6-4b945c26acbe admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:00.945257 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1fb68b-d7c4-4afd-89a6-4b945c26acbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:00.945641 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1fb68b-d7c4-4afd-89a6-4b945c26acbe admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:00.946286 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1fb68b-d7c4-4afd-89a6-4b945c26acbe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:00.947908 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1fb68b-d7c4-4afd-89a6-4b945c26acbe admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:00.953215 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:04:00.959054 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1296/2571] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 24 10:04:00 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:04:00.963165 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1fb68b-d7c4-4afd-89a6-4b945c26acbe admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:00.973041 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1fb68b-d7c4-4afd-89a6-4b945c26acbe admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:00.977978 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df9ef64-423a-4de1-8b80-17d74ac53fb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:00.978629 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df9ef64-423a-4de1-8b80-17d74ac53fb3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:00.978912 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df9ef64-423a-4de1-8b80-17d74ac53fb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:00.979188 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df9ef64-423a-4de1-8b80-17d74ac53fb3 admin admin] PATH_INFO: `/v3/projects/c155eecbc1a84edf9d32431339e53624/users/7084feee261f4255985b6494eacec98a/roles/136cdf78e00a426997dd4e37faff489b` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:00.980795 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3c0e5b-4895-4d47-9c1d-a2dc27464804 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:00.984527 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68aec99-1eac-4207-896a-d2f3466a05b8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c32b83577c46488906fe409a790b24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:00.985162 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68aec99-1eac-4207-896a-d2f3466a05b8 service cinder] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:00.985442 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68aec99-1eac-4207-896a-d2f3466a05b8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:00.985715 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68aec99-1eac-4207-896a-d2f3466a05b8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:00.995428 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68aec99-1eac-4207-896a-d2f3466a05b8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:00.996778 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df9ef64-423a-4de1-8b80-17d74ac53fb3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c155eecbc1a84edf9d32431339e53624, user_id=7084feee261f4255985b6494eacec98a, role_id=136cdf78e00a426997dd4e37faff489b)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:00.996911 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68aec99-1eac-4207-896a-d2f3466a05b8 service cinder] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:00.999453 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df9ef64-423a-4de1-8b80-17d74ac53fb3 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:01.007637 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1276/2572] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 10:04:00 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:04:01.106683 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1297/2573] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 24 10:04:00 2022] PUT /identity/v3/projects/c155eecbc1a84edf9d32431339e53624/users/7084feee261f4255985b6494eacec98a/roles/136cdf78e00a426997dd4e37faff489b => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:04:01.118749 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd63aa45-c05a-470a-8072-f4ed78b304b4 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:01.119150 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd63aa45-c05a-470a-8072-f4ed78b304b4 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:01.119491 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd63aa45-c05a-470a-8072-f4ed78b304b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:01.133641 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df9ef64-423a-4de1-8b80-17d74ac53fb3 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:01.136163 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-2df9ef64-423a-4de1-8b80-17d74ac53fb3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:01.149029 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-dd63aa45-c05a-470a-8072-f4ed78b304b4 None None] MFA Rules not processed for user `7084feee261f4255985b6494eacec98a`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:04:01.150229 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-dd63aa45-c05a-470a-8072-f4ed78b304b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:01.181027 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:04:01.194518 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df9ef64-423a-4de1-8b80-17d74ac53fb3 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:01.198482 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1aa343a-8e9c-4f5e-a7c1-1e0512e28816 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c32b83577c46488906fe409a790b24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:01.199094 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1aa343a-8e9c-4f5e-a7c1-1e0512e28816 service cinder] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:01.199379 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1aa343a-8e9c-4f5e-a7c1-1e0512e28816 service cinder] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:01.199679 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1aa343a-8e9c-4f5e-a7c1-1e0512e28816 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:01.201629 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-c1aa343a-8e9c-4f5e-a7c1-1e0512e28816 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:01.217044 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1277/2574] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 10:04:01 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:04:01.222000 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd63aa45-c05a-470a-8072-f4ed78b304b4 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:01.262077 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1aa343a-8e9c-4f5e-a7c1-1e0512e28816 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:01.263655 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1aa343a-8e9c-4f5e-a7c1-1e0512e28816 service cinder] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:01.272942 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1298/2575] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 10:04:01 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 10:04:01.286544 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd63aa45-c05a-470a-8072-f4ed78b304b4 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:01.291626 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07badfa0-3b72-4f0a-8ab0-1e10b80540c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:01.292292 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07badfa0-3b72-4f0a-8ab0-1e10b80540c2 admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:01.292571 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07badfa0-3b72-4f0a-8ab0-1e10b80540c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:01.292844 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07badfa0-3b72-4f0a-8ab0-1e10b80540c2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:01.293491 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07badfa0-3b72-4f0a-8ab0-1e10b80540c2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:01.295411 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07badfa0-3b72-4f0a-8ab0-1e10b80540c2 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:01.417287 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1278/2576] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 24 10:04:01 2022] POST /identity/v3/projects => generated 401 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:04:01.421376 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1aa343a-8e9c-4f5e-a7c1-1e0512e28816 service cinder] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:01.433121 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1aa343a-8e9c-4f5e-a7c1-1e0512e28816 service cinder] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:01.489814 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b5d302-f9be-4422-9757-d52244a6d8d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:01.490353 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b5d302-f9be-4422-9757-d52244a6d8d3 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:01.490666 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b5d302-f9be-4422-9757-d52244a6d8d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:01.490945 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b5d302-f9be-4422-9757-d52244a6d8d3 admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:01.491572 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b5d302-f9be-4422-9757-d52244a6d8d3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:01.493523 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b5d302-f9be-4422-9757-d52244a6d8d3 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:01.615556 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1299/2577] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 24 10:04:01 2022] POST /identity/v3/users => generated 307 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:04:01.619538 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07badfa0-3b72-4f0a-8ab0-1e10b80540c2 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:01.629666 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07badfa0-3b72-4f0a-8ab0-1e10b80540c2 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:01.634519 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34651f21-8ae0-4cbd-b4fd-3a610c80d918 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:01.635150 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34651f21-8ae0-4cbd-b4fd-3a610c80d918 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:01.635432 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34651f21-8ae0-4cbd-b4fd-3a610c80d918 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:01.640209 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34651f21-8ae0-4cbd-b4fd-3a610c80d918 admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:01.640209 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34651f21-8ae0-4cbd-b4fd-3a610c80d918 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:01.640209 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34651f21-8ae0-4cbd-b4fd-3a610c80d918 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:01.648856 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1279/2578] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 24 10:04:01 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:04:01.653040 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b5d302-f9be-4422-9757-d52244a6d8d3 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:01.664422 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b5d302-f9be-4422-9757-d52244a6d8d3 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:01.670150 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd0aa3e-dd8c-4092-8575-bea73909ecf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:01.673855 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd0aa3e-dd8c-4092-8575-bea73909ecf7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:01.673855 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd0aa3e-dd8c-4092-8575-bea73909ecf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:01.673855 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd0aa3e-dd8c-4092-8575-bea73909ecf7 admin admin] PATH_INFO: `/v3/projects/c61211e2cb1e45db90664e5bc412f2af/users/e8206225fd374b669b2fbe8e4e583d84/roles/15f634d4a2d247ddab90c458e5e1c141` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:01.686787 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd0aa3e-dd8c-4092-8575-bea73909ecf7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c61211e2cb1e45db90664e5bc412f2af, user_id=e8206225fd374b669b2fbe8e4e583d84, role_id=15f634d4a2d247ddab90c458e5e1c141)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:01.689604 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd0aa3e-dd8c-4092-8575-bea73909ecf7 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:01.692641 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:04:01.769409 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1300/2579] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 24 10:04:01 2022] PUT /identity/v3/projects/c61211e2cb1e45db90664e5bc412f2af/users/e8206225fd374b669b2fbe8e4e583d84/roles/15f634d4a2d247ddab90c458e5e1c141 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:04:01.773510 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34651f21-8ae0-4cbd-b4fd-3a610c80d918 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:01.784030 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34651f21-8ae0-4cbd-b4fd-3a610c80d918 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:01.840946 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff97d1f-7cc5-443e-b66b-7d21f32e28c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:01.841553 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff97d1f-7cc5-443e-b66b-7d21f32e28c1 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:01.841834 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff97d1f-7cc5-443e-b66b-7d21f32e28c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:01.842166 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff97d1f-7cc5-443e-b66b-7d21f32e28c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:01.842815 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff97d1f-7cc5-443e-b66b-7d21f32e28c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:01.844430 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff97d1f-7cc5-443e-b66b-7d21f32e28c1 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:01.854613 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1280/2580] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 24 10:04:01 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:04:01.858979 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd0aa3e-dd8c-4092-8575-bea73909ecf7 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:01.869832 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd0aa3e-dd8c-4092-8575-bea73909ecf7 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:01.874863 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b001104-c063-43b7-9a6b-74267384d05d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:01.875571 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b001104-c063-43b7-9a6b-74267384d05d admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:01.875882 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b001104-c063-43b7-9a6b-74267384d05d admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:01.876167 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b001104-c063-43b7-9a6b-74267384d05d admin admin] PATH_INFO: `/v3/projects/c61211e2cb1e45db90664e5bc412f2af/users/e8206225fd374b669b2fbe8e4e583d84/roles/15f634d4a2d247ddab90c458e5e1c141` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:01.879533 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b001104-c063-43b7-9a6b-74267384d05d admin admin] RBAC: Authorizing `identity:create_grant(project_id=c61211e2cb1e45db90664e5bc412f2af, user_id=e8206225fd374b669b2fbe8e4e583d84, role_id=15f634d4a2d247ddab90c458e5e1c141)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:01.883624 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b001104-c063-43b7-9a6b-74267384d05d admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:01.919683 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1301/2581] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 24 10:04:01 2022] PUT /identity/v3/projects/c61211e2cb1e45db90664e5bc412f2af/users/e8206225fd374b669b2fbe8e4e583d84/roles/15f634d4a2d247ddab90c458e5e1c141 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:04:01.929046 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1d9a05-1776-41eb-b752-7aceb3c08187 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:01.929992 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1d9a05-1776-41eb-b752-7aceb3c08187 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:01.930370 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1d9a05-1776-41eb-b752-7aceb3c08187 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:01.962855 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-9e1d9a05-1776-41eb-b752-7aceb3c08187 None None] MFA Rules not processed for user `e8206225fd374b669b2fbe8e4e583d84`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:04:01.964092 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-9e1d9a05-1776-41eb-b752-7aceb3c08187 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:01.994764 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:04:02.035028 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1281/2582] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 10:04:01 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:04:02.046325 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b001104-c063-43b7-9a6b-74267384d05d admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:02.104141 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b001104-c063-43b7-9a6b-74267384d05d admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:02.107877 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f937d508-8235-44c5-9ab6-12a6689d7a6c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c32b83577c46488906fe409a790b24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:02.108572 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f937d508-8235-44c5-9ab6-12a6689d7a6c service cinder] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:02.108855 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f937d508-8235-44c5-9ab6-12a6689d7a6c service cinder] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:02.109135 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f937d508-8235-44c5-9ab6-12a6689d7a6c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:02.118834 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f937d508-8235-44c5-9ab6-12a6689d7a6c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:02.120300 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f937d508-8235-44c5-9ab6-12a6689d7a6c service cinder] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:02.129874 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1302/2583] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 10:04:02 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:04:02.345193 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1d9a05-1776-41eb-b752-7aceb3c08187 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:02.355675 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1d9a05-1776-41eb-b752-7aceb3c08187 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:02.359219 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18f0136-c09b-4a84-b548-9cb8e7ffbb79 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c32b83577c46488906fe409a790b24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:02.359885 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18f0136-c09b-4a84-b548-9cb8e7ffbb79 service cinder] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:02.360189 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18f0136-c09b-4a84-b548-9cb8e7ffbb79 service cinder] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:02.360457 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18f0136-c09b-4a84-b548-9cb8e7ffbb79 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:02.416096 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18f0136-c09b-4a84-b548-9cb8e7ffbb79 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:02.417539 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18f0136-c09b-4a84-b548-9cb8e7ffbb79 service cinder] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:02.426317 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1282/2584] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 10:04:02 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:04:06.208724 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f937d508-8235-44c5-9ab6-12a6689d7a6c service cinder] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:06.284430 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f937d508-8235-44c5-9ab6-12a6689d7a6c service cinder] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:06.291123 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9138bc49-f44b-4133-8c83-5880407fe938 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:06.292136 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9138bc49-f44b-4133-8c83-5880407fe938 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:06.292572 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9138bc49-f44b-4133-8c83-5880407fe938 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:06.293033 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9138bc49-f44b-4133-8c83-5880407fe938 admin admin] PATH_INFO: `/v3/users/7c280ab9e891443a91c6861a89890e9f` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:06.294332 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9138bc49-f44b-4133-8c83-5880407fe938 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7c280ab9e891443a91c6861a89890e9f)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:06.296432 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9138bc49-f44b-4133-8c83-5880407fe938 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:06.468509 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-9138bc49-f44b-4133-8c83-5880407fe938 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7c280ab9e891443a91c6861a89890e9f'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:06.469289 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-9138bc49-f44b-4133-8c83-5880407fe938 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7c280ab9e891443a91c6861a89890e9f'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:06.533979 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-9138bc49-f44b-4133-8c83-5880407fe938 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7c280ab9e891443a91c6861a89890e9f'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:06.534622 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-9138bc49-f44b-4133-8c83-5880407fe938 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7c280ab9e891443a91c6861a89890e9f'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:06.543303 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-9138bc49-f44b-4133-8c83-5880407fe938 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7c280ab9e891443a91c6861a89890e9f'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:06.564405 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1303/2585] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 24 10:04:06 2022] DELETE /identity/v3/users/7c280ab9e891443a91c6861a89890e9f => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:04:06.569300 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5aa9c7-fbd5-48d9-bd1c-099f6549163c None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:06.572481 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5aa9c7-fbd5-48d9-bd1c-099f6549163c None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:06.572481 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5aa9c7-fbd5-48d9-bd1c-099f6549163c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:06.590202 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-8c5aa9c7-fbd5-48d9-bd1c-099f6549163c None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:04:06.591475 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-8c5aa9c7-fbd5-48d9-bd1c-099f6549163c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:06.656839 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1283/2586] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 10:04:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:04:06.667174 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9138bc49-f44b-4133-8c83-5880407fe938 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:06.668681 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-9138bc49-f44b-4133-8c83-5880407fe938 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:06.675318 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:04:06.740850 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9138bc49-f44b-4133-8c83-5880407fe938 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:06.745752 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab6e60a-09a0-42f1-b10a-922b88c438d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e5260e798404e8bb12b25ca8ea97052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:06.746362 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab6e60a-09a0-42f1-b10a-922b88c438d3 service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:06.746639 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab6e60a-09a0-42f1-b10a-922b88c438d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:06.746917 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab6e60a-09a0-42f1-b10a-922b88c438d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:06.756993 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab6e60a-09a0-42f1-b10a-922b88c438d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:06.758433 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab6e60a-09a0-42f1-b10a-922b88c438d3 service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:06.770760 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1304/2587] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 10:04:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:04:07.244310 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5aa9c7-fbd5-48d9-bd1c-099f6549163c None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:07.245741 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-8c5aa9c7-fbd5-48d9-bd1c-099f6549163c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:07.258470 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5aa9c7-fbd5-48d9-bd1c-099f6549163c None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:07.263273 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7c9a19-ef55-4cc0-9572-051bdccea2ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:07.264088 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7c9a19-ef55-4cc0-9572-051bdccea2ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:07.264327 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7c9a19-ef55-4cc0-9572-051bdccea2ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:07.264605 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7c9a19-ef55-4cc0-9572-051bdccea2ec admin admin] PATH_INFO: `/v3/projects/f5e76a8ec44d48a4a998705a74b36f76` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:07.265798 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7c9a19-ef55-4cc0-9572-051bdccea2ec admin admin] RBAC: Authorizing `identity:delete_project(project_id=f5e76a8ec44d48a4a998705a74b36f76)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:07.267703 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7c9a19-ef55-4cc0-9572-051bdccea2ec admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:07.404560 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-ab7c9a19-ef55-4cc0-9572-051bdccea2ec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f5e76a8ec44d48a4a998705a74b36f76', 'request_id': 'req-ab7c9a19-ef55-4cc0-9572-051bdccea2ec'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:07.410792 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-ab7c9a19-ef55-4cc0-9572-051bdccea2ec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f5e76a8ec44d48a4a998705a74b36f76', 'request_id': 'req-ab7c9a19-ef55-4cc0-9572-051bdccea2ec'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:07.414805 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:04:07.417010 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-ab7c9a19-ef55-4cc0-9572-051bdccea2ec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f5e76a8ec44d48a4a998705a74b36f76', 'request_id': 'req-ab7c9a19-ef55-4cc0-9572-051bdccea2ec'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:07.534026 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-ab7c9a19-ef55-4cc0-9572-051bdccea2ec admin admin] The token cache is being invalidate because project f5e76a8ec44d48a4a998705a74b36f76 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:04:07.534345 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-ab7c9a19-ef55-4cc0-9572-051bdccea2ec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:07.534995 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-ab7c9a19-ef55-4cc0-9572-051bdccea2ec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:07.536532 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1284/2588] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 24 10:04:07 2022] DELETE /identity/v3/projects/f5e76a8ec44d48a4a998705a74b36f76 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:04:07.540468 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab6e60a-09a0-42f1-b10a-922b88c438d3 service neutron] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:07.550012 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab6e60a-09a0-42f1-b10a-922b88c438d3 service neutron] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:07.605270 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f63bbb4-8830-479f-bb21-163d337b3cdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:07.605971 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f63bbb4-8830-479f-bb21-163d337b3cdf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:07.606245 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f63bbb4-8830-479f-bb21-163d337b3cdf admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:07.606514 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f63bbb4-8830-479f-bb21-163d337b3cdf admin admin] PATH_INFO: `/v3/users/7ff7d3b8e0c346ba8257e39d45998ba9` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:07.607626 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f63bbb4-8830-479f-bb21-163d337b3cdf admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ff7d3b8e0c346ba8257e39d45998ba9)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:07.609555 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f63bbb4-8830-479f-bb21-163d337b3cdf admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:07.775650 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-9f63bbb4-8830-479f-bb21-163d337b3cdf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ff7d3b8e0c346ba8257e39d45998ba9'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:07.776415 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-9f63bbb4-8830-479f-bb21-163d337b3cdf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ff7d3b8e0c346ba8257e39d45998ba9'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:07.847976 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-9f63bbb4-8830-479f-bb21-163d337b3cdf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ff7d3b8e0c346ba8257e39d45998ba9'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:07.848802 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-9f63bbb4-8830-479f-bb21-163d337b3cdf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ff7d3b8e0c346ba8257e39d45998ba9'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:07.857313 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-9f63bbb4-8830-479f-bb21-163d337b3cdf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ff7d3b8e0c346ba8257e39d45998ba9'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:07.876572 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1305/2589] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 24 10:04:07 2022] DELETE /identity/v3/users/7ff7d3b8e0c346ba8257e39d45998ba9 => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:04:08.686981 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7c9a19-ef55-4cc0-9572-051bdccea2ec admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:08.688699 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-ab7c9a19-ef55-4cc0-9572-051bdccea2ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:08.702610 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7c9a19-ef55-4cc0-9572-051bdccea2ec admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:08.763037 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a712acd-ec3b-45bf-acc6-1f115adc4138 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:08.763713 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a712acd-ec3b-45bf-acc6-1f115adc4138 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:08.763999 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a712acd-ec3b-45bf-acc6-1f115adc4138 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:08.764277 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a712acd-ec3b-45bf-acc6-1f115adc4138 admin admin] PATH_INFO: `/v3/projects/0074f864f9884477803dd6b1ce858b00` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:08.765407 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a712acd-ec3b-45bf-acc6-1f115adc4138 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0074f864f9884477803dd6b1ce858b00)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:08.767346 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a712acd-ec3b-45bf-acc6-1f115adc4138 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:08.884162 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-1a712acd-ec3b-45bf-acc6-1f115adc4138 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0074f864f9884477803dd6b1ce858b00', 'request_id': 'req-1a712acd-ec3b-45bf-acc6-1f115adc4138'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:08.890218 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-1a712acd-ec3b-45bf-acc6-1f115adc4138 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0074f864f9884477803dd6b1ce858b00', 'request_id': 'req-1a712acd-ec3b-45bf-acc6-1f115adc4138'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:08.896327 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-1a712acd-ec3b-45bf-acc6-1f115adc4138 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0074f864f9884477803dd6b1ce858b00', 'request_id': 'req-1a712acd-ec3b-45bf-acc6-1f115adc4138'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:08.985798 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-1a712acd-ec3b-45bf-acc6-1f115adc4138 admin admin] The token cache is being invalidate because project 0074f864f9884477803dd6b1ce858b00 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:04:08.986040 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-1a712acd-ec3b-45bf-acc6-1f115adc4138 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:08.986674 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-1a712acd-ec3b-45bf-acc6-1f115adc4138 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:08.988299 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1285/2590] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 24 10:04:08 2022] DELETE /identity/v3/projects/0074f864f9884477803dd6b1ce858b00 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:04:08.993100 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec5d0dd-a0cc-4a69-865c-baf278fc7f52 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:08.993603 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec5d0dd-a0cc-4a69-865c-baf278fc7f52 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:08.993700 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec5d0dd-a0cc-4a69-865c-baf278fc7f52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:09.013207 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-5ec5d0dd-a0cc-4a69-865c-baf278fc7f52 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:04:09.014546 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-5ec5d0dd-a0cc-4a69-865c-baf278fc7f52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:09.036112 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:04:09.074975 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1306/2591] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 10:04:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:04:09.084369 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dfba02-ec4c-41a5-b6ee-8f51279ab02c None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:09.084655 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dfba02-ec4c-41a5-b6ee-8f51279ab02c None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:09.084929 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dfba02-ec4c-41a5-b6ee-8f51279ab02c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:09.104890 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-a5dfba02-ec4c-41a5-b6ee-8f51279ab02c None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:04:09.106125 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-a5dfba02-ec4c-41a5-b6ee-8f51279ab02c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:09.114098 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1286/2592] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 10:04:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:04:09.118437 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec5d0dd-a0cc-4a69-865c-baf278fc7f52 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:09.128430 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec5d0dd-a0cc-4a69-865c-baf278fc7f52 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:09.134318 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff202974-25fd-4b1d-9111-08dad799eb5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:09.134958 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff202974-25fd-4b1d-9111-08dad799eb5e admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:09.135234 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff202974-25fd-4b1d-9111-08dad799eb5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:09.135503 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff202974-25fd-4b1d-9111-08dad799eb5e admin admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:09.136200 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff202974-25fd-4b1d-9111-08dad799eb5e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:04:09.136488 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff202974-25fd-4b1d-9111-08dad799eb5e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:09.138089 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff202974-25fd-4b1d-9111-08dad799eb5e admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:09.151888 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1307/2593] 10.222.0.16 () {58 vars in 1153 bytes} [Wed Aug 24 10:04:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:04:09.157213 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dfba02-ec4c-41a5-b6ee-8f51279ab02c None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:09.167057 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dfba02-ec4c-41a5-b6ee-8f51279ab02c None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:09.172081 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abdb5ad-b65a-42cb-860c-cf13e61d3d9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:09.172689 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abdb5ad-b65a-42cb-860c-cf13e61d3d9d admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:09.172998 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abdb5ad-b65a-42cb-860c-cf13e61d3d9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:09.173273 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abdb5ad-b65a-42cb-860c-cf13e61d3d9d admin admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:09.173945 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abdb5ad-b65a-42cb-860c-cf13e61d3d9d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:09.175876 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abdb5ad-b65a-42cb-860c-cf13e61d3d9d admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:09.279316 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1287/2594] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 24 10:04:09 2022] POST /identity/v3/projects => generated 377 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:04:09.280639 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff202974-25fd-4b1d-9111-08dad799eb5e admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:09.293515 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff202974-25fd-4b1d-9111-08dad799eb5e admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:09.361792 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc68dc1e-9e97-4fe2-a637-415ff9e29c24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:09.362399 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc68dc1e-9e97-4fe2-a637-415ff9e29c24 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:09.362684 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc68dc1e-9e97-4fe2-a637-415ff9e29c24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:09.362956 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc68dc1e-9e97-4fe2-a637-415ff9e29c24 admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:09.363618 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc68dc1e-9e97-4fe2-a637-415ff9e29c24 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:09.365524 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc68dc1e-9e97-4fe2-a637-415ff9e29c24 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:09.483872 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1308/2595] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 24 10:04:09 2022] POST /identity/v3/users => generated 289 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:04:09.488055 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abdb5ad-b65a-42cb-860c-cf13e61d3d9d admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:09.498056 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abdb5ad-b65a-42cb-860c-cf13e61d3d9d admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:09.502928 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2dfd74b-3be7-467e-877c-14f4d768f24e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:09.503534 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dfd74b-3be7-467e-877c-14f4d768f24e admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:09.503848 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dfd74b-3be7-467e-877c-14f4d768f24e admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:09.504126 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dfd74b-3be7-467e-877c-14f4d768f24e admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:09.504771 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2dfd74b-3be7-467e-877c-14f4d768f24e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:09.506902 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2dfd74b-3be7-467e-877c-14f4d768f24e admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:09.518922 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1288/2596] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 24 10:04:09 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:04:09.521544 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc68dc1e-9e97-4fe2-a637-415ff9e29c24 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:09.531478 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc68dc1e-9e97-4fe2-a637-415ff9e29c24 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:09.536435 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9c3ba7-c064-483e-ac5c-8349c330a107 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:09.541951 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9c3ba7-c064-483e-ac5c-8349c330a107 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:09.541951 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9c3ba7-c064-483e-ac5c-8349c330a107 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:09.541951 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9c3ba7-c064-483e-ac5c-8349c330a107 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:09.541951 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9c3ba7-c064-483e-ac5c-8349c330a107 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:09.541951 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9c3ba7-c064-483e-ac5c-8349c330a107 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:09.550324 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1309/2597] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 24 10:04:09 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:04:09.554374 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2dfd74b-3be7-467e-877c-14f4d768f24e admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:09.564253 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2dfd74b-3be7-467e-877c-14f4d768f24e admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:09.569431 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700567f9-f2fa-4bcf-8a50-e19bbe0edcb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:09.570648 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700567f9-f2fa-4bcf-8a50-e19bbe0edcb9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:09.570648 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700567f9-f2fa-4bcf-8a50-e19bbe0edcb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:09.570648 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700567f9-f2fa-4bcf-8a50-e19bbe0edcb9 admin admin] PATH_INFO: `/v3/projects/2e31467031a54416b979d1099b610f26/users/3f8b99f006e84eefa53dc254c07c4418/roles/136cdf78e00a426997dd4e37faff489b` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:09.585774 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700567f9-f2fa-4bcf-8a50-e19bbe0edcb9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e31467031a54416b979d1099b610f26, user_id=3f8b99f006e84eefa53dc254c07c4418, role_id=136cdf78e00a426997dd4e37faff489b)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:09.588466 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700567f9-f2fa-4bcf-8a50-e19bbe0edcb9 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:09.689222 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1289/2598] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 24 10:04:09 2022] PUT /identity/v3/projects/2e31467031a54416b979d1099b610f26/users/3f8b99f006e84eefa53dc254c07c4418/roles/136cdf78e00a426997dd4e37faff489b => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 10:04:09.700009 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5f345c-5a45-4f75-9abe-1730fe2d2c1f None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:09.700272 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5f345c-5a45-4f75-9abe-1730fe2d2c1f None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:09.700542 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5f345c-5a45-4f75-9abe-1730fe2d2c1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:09.728178 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-7b5f345c-5a45-4f75-9abe-1730fe2d2c1f None None] MFA Rules not processed for user `3f8b99f006e84eefa53dc254c07c4418`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:04:09.729413 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-7b5f345c-5a45-4f75-9abe-1730fe2d2c1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:09.792618 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1310/2599] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 10:04:09 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:04:09.797215 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700567f9-f2fa-4bcf-8a50-e19bbe0edcb9 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:09.810869 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:DELETE=9,keystone:SELECT=87,keystone:INSERT=4 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:04:09.858382 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700567f9-f2fa-4bcf-8a50-e19bbe0edcb9 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:09.863098 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464d0bfc-6fa2-4495-a8c6-58a2b699051a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:09.863815 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464d0bfc-6fa2-4495-a8c6-58a2b699051a admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:09.864105 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464d0bfc-6fa2-4495-a8c6-58a2b699051a admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:09.864380 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464d0bfc-6fa2-4495-a8c6-58a2b699051a admin admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:09.865504 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464d0bfc-6fa2-4495-a8c6-58a2b699051a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:09.867549 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464d0bfc-6fa2-4495-a8c6-58a2b699051a admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:09.935916 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1290/2600] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 24 10:04:09 2022] POST /identity/v3/projects => generated 377 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 10:04:09.939883 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5f345c-5a45-4f75-9abe-1730fe2d2c1f None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:09.950957 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5f345c-5a45-4f75-9abe-1730fe2d2c1f None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:09.969385 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:04:10.007635 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e389b7-df03-480a-9bc0-fd8fe8319088 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:10.008407 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e389b7-df03-480a-9bc0-fd8fe8319088 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:10.008535 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e389b7-df03-480a-9bc0-fd8fe8319088 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:10.008803 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e389b7-df03-480a-9bc0-fd8fe8319088 admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:10.009429 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e389b7-df03-480a-9bc0-fd8fe8319088 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:10.011326 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e389b7-df03-480a-9bc0-fd8fe8319088 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:10.077850 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1311/2601] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 24 10:04:09 2022] POST /identity/v3/users => generated 289 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 10:04:10.082053 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464d0bfc-6fa2-4495-a8c6-58a2b699051a admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:10.092322 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464d0bfc-6fa2-4495-a8c6-58a2b699051a admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:10.096857 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd105d5-b3b6-415a-a425-656dc59dde9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:10.097451 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd105d5-b3b6-415a-a425-656dc59dde9d admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:10.097732 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd105d5-b3b6-415a-a425-656dc59dde9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:10.098005 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd105d5-b3b6-415a-a425-656dc59dde9d admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:10.098663 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd105d5-b3b6-415a-a425-656dc59dde9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:10.100899 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd105d5-b3b6-415a-a425-656dc59dde9d admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:10.111370 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1291/2602] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 24 10:04:10 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:04:10.115972 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e389b7-df03-480a-9bc0-fd8fe8319088 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:10.126479 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e389b7-df03-480a-9bc0-fd8fe8319088 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:10.131494 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e0f206-4fec-4cd4-8d0e-214e9dad3b2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:10.132139 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e0f206-4fec-4cd4-8d0e-214e9dad3b2c admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:10.132422 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e0f206-4fec-4cd4-8d0e-214e9dad3b2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:10.132698 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e0f206-4fec-4cd4-8d0e-214e9dad3b2c admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:10.133345 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e0f206-4fec-4cd4-8d0e-214e9dad3b2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:10.134943 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e0f206-4fec-4cd4-8d0e-214e9dad3b2c admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:10.144886 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1312/2603] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 24 10:04:10 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:04:10.149038 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd105d5-b3b6-415a-a425-656dc59dde9d admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:10.158795 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd105d5-b3b6-415a-a425-656dc59dde9d admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:10.163874 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6d5716-b2e2-4fe1-b918-ada2bfc03e29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:10.164519 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6d5716-b2e2-4fe1-b918-ada2bfc03e29 admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:10.164803 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6d5716-b2e2-4fe1-b918-ada2bfc03e29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:10.165075 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6d5716-b2e2-4fe1-b918-ada2bfc03e29 admin admin] PATH_INFO: `/v3/projects/5f6f30d09d52449ab0cf34bc9c1b47b1/users/879bf2c89c4c4309a240a51582a78ba6/roles/136cdf78e00a426997dd4e37faff489b` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:10.180136 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6d5716-b2e2-4fe1-b918-ada2bfc03e29 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f6f30d09d52449ab0cf34bc9c1b47b1, user_id=879bf2c89c4c4309a240a51582a78ba6, role_id=136cdf78e00a426997dd4e37faff489b)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:10.182774 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6d5716-b2e2-4fe1-b918-ada2bfc03e29 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:10.266487 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1292/2604] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 24 10:04:10 2022] PUT /identity/v3/projects/5f6f30d09d52449ab0cf34bc9c1b47b1/users/879bf2c89c4c4309a240a51582a78ba6/roles/136cdf78e00a426997dd4e37faff489b => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:04:10.277761 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eeca3db-364e-4e31-abb1-ed76115e1f56 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:10.278074 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eeca3db-364e-4e31-abb1-ed76115e1f56 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:10.278354 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eeca3db-364e-4e31-abb1-ed76115e1f56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:10.310547 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-4eeca3db-364e-4e31-abb1-ed76115e1f56 None None] MFA Rules not processed for user `879bf2c89c4c4309a240a51582a78ba6`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:04:10.312851 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-4eeca3db-364e-4e31-abb1-ed76115e1f56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:10.388574 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1313/2605] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 10:04:10 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:04:10.392604 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6d5716-b2e2-4fe1-b918-ada2bfc03e29 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:10.454998 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6d5716-b2e2-4fe1-b918-ada2bfc03e29 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:10.460133 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdfd995-c748-4c59-93bb-fc6568de3e6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:10.460738 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdfd995-c748-4c59-93bb-fc6568de3e6c admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:10.461019 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdfd995-c748-4c59-93bb-fc6568de3e6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:10.461293 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdfd995-c748-4c59-93bb-fc6568de3e6c admin admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:10.461925 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdfd995-c748-4c59-93bb-fc6568de3e6c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:10.463849 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdfd995-c748-4c59-93bb-fc6568de3e6c admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:10.580562 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1293/2606] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 24 10:04:10 2022] POST /identity/v3/projects => generated 379 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 10:04:10.584922 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eeca3db-364e-4e31-abb1-ed76115e1f56 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:10.595690 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eeca3db-364e-4e31-abb1-ed76115e1f56 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:10.651785 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e6e291-4398-46d3-8294-e4cb69e4b4a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:10.652387 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e6e291-4398-46d3-8294-e4cb69e4b4a4 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:10.652662 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e6e291-4398-46d3-8294-e4cb69e4b4a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:10.652936 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e6e291-4398-46d3-8294-e4cb69e4b4a4 admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:10.653559 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e6e291-4398-46d3-8294-e4cb69e4b4a4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:10.655456 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e6e291-4398-46d3-8294-e4cb69e4b4a4 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:10.740414 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1314/2607] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 24 10:04:10 2022] POST /identity/v3/users => generated 296 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:04:10.744342 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdfd995-c748-4c59-93bb-fc6568de3e6c admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:10.754398 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdfd995-c748-4c59-93bb-fc6568de3e6c admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:10.759460 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54112f6-28be-4daa-bc94-6009a953b97a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:10.760077 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54112f6-28be-4daa-bc94-6009a953b97a admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:10.760357 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54112f6-28be-4daa-bc94-6009a953b97a admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:10.760629 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54112f6-28be-4daa-bc94-6009a953b97a admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:10.761261 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54112f6-28be-4daa-bc94-6009a953b97a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:10.762846 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54112f6-28be-4daa-bc94-6009a953b97a admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:10.773899 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1294/2608] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 24 10:04:10 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:04:10.777992 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e6e291-4398-46d3-8294-e4cb69e4b4a4 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:10.787912 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e6e291-4398-46d3-8294-e4cb69e4b4a4 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:10.792502 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961b5bed-a261-44ac-9dd8-a2089c7907c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:10.793169 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961b5bed-a261-44ac-9dd8-a2089c7907c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:10.793451 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961b5bed-a261-44ac-9dd8-a2089c7907c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:10.793722 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961b5bed-a261-44ac-9dd8-a2089c7907c6 admin admin] PATH_INFO: `/v3/projects/6802fafdf54247fa9d0a495c26809ba4/users/aad8f8b8a6cf48019b7f987bd5d1cd3c/roles/15f634d4a2d247ddab90c458e5e1c141` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:10.808077 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961b5bed-a261-44ac-9dd8-a2089c7907c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6802fafdf54247fa9d0a495c26809ba4, user_id=aad8f8b8a6cf48019b7f987bd5d1cd3c, role_id=15f634d4a2d247ddab90c458e5e1c141)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:10.810790 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961b5bed-a261-44ac-9dd8-a2089c7907c6 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:10.813755 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:04:10.906710 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1315/2609] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 24 10:04:10 2022] PUT /identity/v3/projects/6802fafdf54247fa9d0a495c26809ba4/users/aad8f8b8a6cf48019b7f987bd5d1cd3c/roles/15f634d4a2d247ddab90c458e5e1c141 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:04:10.910609 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54112f6-28be-4daa-bc94-6009a953b97a admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:10.920471 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54112f6-28be-4daa-bc94-6009a953b97a admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:10.952845 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:04:10.976127 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb695063-e46b-449e-b39d-809aad149247 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:10.976736 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb695063-e46b-449e-b39d-809aad149247 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:10.977021 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb695063-e46b-449e-b39d-809aad149247 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:10.977302 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb695063-e46b-449e-b39d-809aad149247 admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:10.977980 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb695063-e46b-449e-b39d-809aad149247 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:10.979917 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb695063-e46b-449e-b39d-809aad149247 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:10.989926 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1295/2610] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 24 10:04:10 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:04:10.994090 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961b5bed-a261-44ac-9dd8-a2089c7907c6 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:11.004171 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961b5bed-a261-44ac-9dd8-a2089c7907c6 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:11.009230 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566d9f1e-6125-407a-81c8-3bacdb7cf154 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:11.009870 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566d9f1e-6125-407a-81c8-3bacdb7cf154 admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:11.010146 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566d9f1e-6125-407a-81c8-3bacdb7cf154 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:11.010417 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566d9f1e-6125-407a-81c8-3bacdb7cf154 admin admin] PATH_INFO: `/v3/projects/6802fafdf54247fa9d0a495c26809ba4/users/aad8f8b8a6cf48019b7f987bd5d1cd3c/roles/15f634d4a2d247ddab90c458e5e1c141` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:11.012372 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566d9f1e-6125-407a-81c8-3bacdb7cf154 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6802fafdf54247fa9d0a495c26809ba4, user_id=aad8f8b8a6cf48019b7f987bd5d1cd3c, role_id=15f634d4a2d247ddab90c458e5e1c141)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:11.014983 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566d9f1e-6125-407a-81c8-3bacdb7cf154 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:11.025619 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1316/2611] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 24 10:04:10 2022] PUT /identity/v3/projects/6802fafdf54247fa9d0a495c26809ba4/users/aad8f8b8a6cf48019b7f987bd5d1cd3c/roles/15f634d4a2d247ddab90c458e5e1c141 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:04:11.035756 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b210c66d-e4f3-4b3f-8dcb-d2e1a99026e8 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:11.036047 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b210c66d-e4f3-4b3f-8dcb-d2e1a99026e8 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:11.036323 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b210c66d-e4f3-4b3f-8dcb-d2e1a99026e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:11.062960 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-b210c66d-e4f3-4b3f-8dcb-d2e1a99026e8 None None] MFA Rules not processed for user `aad8f8b8a6cf48019b7f987bd5d1cd3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:04:11.064195 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-b210c66d-e4f3-4b3f-8dcb-d2e1a99026e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:11.132760 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1296/2612] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 10:04:11 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:04:11.143930 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566d9f1e-6125-407a-81c8-3bacdb7cf154 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:11.145595 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-566d9f1e-6125-407a-81c8-3bacdb7cf154 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:11.209181 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566d9f1e-6125-407a-81c8-3bacdb7cf154 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:11.212742 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298c4f88-097a-4d14-9fdb-056893ee7166 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c32b83577c46488906fe409a790b24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:11.213341 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298c4f88-097a-4d14-9fdb-056893ee7166 service cinder] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:11.213617 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298c4f88-097a-4d14-9fdb-056893ee7166 service cinder] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:11.213886 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298c4f88-097a-4d14-9fdb-056893ee7166 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:11.222626 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298c4f88-097a-4d14-9fdb-056893ee7166 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:11.224085 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298c4f88-097a-4d14-9fdb-056893ee7166 service cinder] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:11.233186 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1317/2613] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 10:04:11 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:04:11.330111 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b210c66d-e4f3-4b3f-8dcb-d2e1a99026e8 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:11.331597 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-b210c66d-e4f3-4b3f-8dcb-d2e1a99026e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:11.347404 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b210c66d-e4f3-4b3f-8dcb-d2e1a99026e8 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:11.350849 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ac7740-f523-46e4-a4f1-034562cf9a23 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c32b83577c46488906fe409a790b24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:11.351478 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ac7740-f523-46e4-a4f1-034562cf9a23 service cinder] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:11.351849 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ac7740-f523-46e4-a4f1-034562cf9a23 service cinder] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:11.352138 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ac7740-f523-46e4-a4f1-034562cf9a23 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:11.411865 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ac7740-f523-46e4-a4f1-034562cf9a23 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:11.413301 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ac7740-f523-46e4-a4f1-034562cf9a23 service cinder] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:11.423696 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1297/2614] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 10:04:11 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:04:13.833242 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298c4f88-097a-4d14-9fdb-056893ee7166 service cinder] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:13.834697 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-298c4f88-097a-4d14-9fdb-056893ee7166 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:13.903557 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298c4f88-097a-4d14-9fdb-056893ee7166 service cinder] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:13.908314 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd2877a-aaf9-4f79-a467-108a4265af8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:13.908891 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd2877a-aaf9-4f79-a467-108a4265af8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:13.909167 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd2877a-aaf9-4f79-a467-108a4265af8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:13.909443 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd2877a-aaf9-4f79-a467-108a4265af8c admin admin] PATH_INFO: `/v3/users/7084feee261f4255985b6494eacec98a` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:13.910571 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd2877a-aaf9-4f79-a467-108a4265af8c admin admin] RBAC: Authorizing `identity:delete_user(user_id=7084feee261f4255985b6494eacec98a)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:13.913070 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd2877a-aaf9-4f79-a467-108a4265af8c admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:14.108301 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-edd2877a-aaf9-4f79-a467-108a4265af8c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7084feee261f4255985b6494eacec98a'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:14.108963 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-edd2877a-aaf9-4f79-a467-108a4265af8c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7084feee261f4255985b6494eacec98a'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:14.186651 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-edd2877a-aaf9-4f79-a467-108a4265af8c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7084feee261f4255985b6494eacec98a'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:14.187200 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-edd2877a-aaf9-4f79-a467-108a4265af8c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7084feee261f4255985b6494eacec98a'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:14.194483 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-edd2877a-aaf9-4f79-a467-108a4265af8c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7084feee261f4255985b6494eacec98a'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:14.211944 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1318/2615] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 24 10:04:13 2022] DELETE /identity/v3/users/7084feee261f4255985b6494eacec98a => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:04:14.216366 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e330bc6-b81b-4f5f-a31f-2185934fd8e5 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:14.216839 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e330bc6-b81b-4f5f-a31f-2185934fd8e5 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:14.216954 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e330bc6-b81b-4f5f-a31f-2185934fd8e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:14.236415 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-7e330bc6-b81b-4f5f-a31f-2185934fd8e5 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:04:14.237686 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-7e330bc6-b81b-4f5f-a31f-2185934fd8e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:14.286032 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:04:14.295660 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1298/2616] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 10:04:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:04:14.305355 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd2877a-aaf9-4f79-a467-108a4265af8c admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:14.306814 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-edd2877a-aaf9-4f79-a467-108a4265af8c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:14.323825 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:04:14.366807 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd2877a-aaf9-4f79-a467-108a4265af8c admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:14.370469 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf2d4a8-04dd-43cc-8125-493964b95583 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e5260e798404e8bb12b25ca8ea97052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:14.372222 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf2d4a8-04dd-43cc-8125-493964b95583 service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:14.372222 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf2d4a8-04dd-43cc-8125-493964b95583 service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:14.372222 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf2d4a8-04dd-43cc-8125-493964b95583 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:14.382265 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf2d4a8-04dd-43cc-8125-493964b95583 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:14.383752 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf2d4a8-04dd-43cc-8125-493964b95583 service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:14.394065 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1319/2617] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 10:04:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:04:14.846598 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e330bc6-b81b-4f5f-a31f-2185934fd8e5 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:14.848219 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-7e330bc6-b81b-4f5f-a31f-2185934fd8e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:14.874092 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e330bc6-b81b-4f5f-a31f-2185934fd8e5 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:14.878805 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddd0805-0b32-486a-ba4a-86981753426b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:14.879438 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddd0805-0b32-486a-ba4a-86981753426b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:14.879751 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddd0805-0b32-486a-ba4a-86981753426b admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:14.880027 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddd0805-0b32-486a-ba4a-86981753426b admin admin] PATH_INFO: `/v3/projects/c155eecbc1a84edf9d32431339e53624` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:14.881187 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddd0805-0b32-486a-ba4a-86981753426b admin admin] RBAC: Authorizing `identity:delete_project(project_id=c155eecbc1a84edf9d32431339e53624)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:14.883086 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddd0805-0b32-486a-ba4a-86981753426b admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:15.002856 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-4ddd0805-0b32-486a-ba4a-86981753426b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c155eecbc1a84edf9d32431339e53624', 'request_id': 'req-4ddd0805-0b32-486a-ba4a-86981753426b'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:15.009498 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-4ddd0805-0b32-486a-ba4a-86981753426b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c155eecbc1a84edf9d32431339e53624', 'request_id': 'req-4ddd0805-0b32-486a-ba4a-86981753426b'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:15.018821 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-4ddd0805-0b32-486a-ba4a-86981753426b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c155eecbc1a84edf9d32431339e53624', 'request_id': 'req-4ddd0805-0b32-486a-ba4a-86981753426b'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:15.133967 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-4ddd0805-0b32-486a-ba4a-86981753426b admin admin] The token cache is being invalidate because project c155eecbc1a84edf9d32431339e53624 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:04:15.138159 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-4ddd0805-0b32-486a-ba4a-86981753426b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:15.138159 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-4ddd0805-0b32-486a-ba4a-86981753426b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:15.138159 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1299/2618] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 24 10:04:14 2022] DELETE /identity/v3/projects/c155eecbc1a84edf9d32431339e53624 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:04:15.141081 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf2d4a8-04dd-43cc-8125-493964b95583 service neutron] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:15.151595 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf2d4a8-04dd-43cc-8125-493964b95583 service neutron] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:15.212784 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abcda43-f298-4f6a-be99-7ea3d3e551ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:15.213537 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abcda43-f298-4f6a-be99-7ea3d3e551ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:15.213824 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abcda43-f298-4f6a-be99-7ea3d3e551ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:15.214102 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abcda43-f298-4f6a-be99-7ea3d3e551ec admin admin] PATH_INFO: `/v3/users/e8206225fd374b669b2fbe8e4e583d84` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:15.215682 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abcda43-f298-4f6a-be99-7ea3d3e551ec admin admin] RBAC: Authorizing `identity:delete_user(user_id=e8206225fd374b669b2fbe8e4e583d84)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:15.217610 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abcda43-f298-4f6a-be99-7ea3d3e551ec admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:15.481724 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-2abcda43-f298-4f6a-be99-7ea3d3e551ec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e8206225fd374b669b2fbe8e4e583d84'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:15.482490 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-2abcda43-f298-4f6a-be99-7ea3d3e551ec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e8206225fd374b669b2fbe8e4e583d84'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:15.585617 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-2abcda43-f298-4f6a-be99-7ea3d3e551ec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e8206225fd374b669b2fbe8e4e583d84'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:15.586341 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-2abcda43-f298-4f6a-be99-7ea3d3e551ec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e8206225fd374b669b2fbe8e4e583d84'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:15.594116 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-2abcda43-f298-4f6a-be99-7ea3d3e551ec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e8206225fd374b669b2fbe8e4e583d84'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:15.611283 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1320/2619] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 24 10:04:15 2022] DELETE /identity/v3/users/e8206225fd374b669b2fbe8e4e583d84 => generated 0 bytes in 471 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:04:16.010760 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddd0805-0b32-486a-ba4a-86981753426b admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:16.012273 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-4ddd0805-0b32-486a-ba4a-86981753426b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:16.029042 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddd0805-0b32-486a-ba4a-86981753426b admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:16.086789 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c393504d-63b7-4729-8c84-2aa2503b7ff6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:16.087445 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c393504d-63b7-4729-8c84-2aa2503b7ff6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:16.087760 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c393504d-63b7-4729-8c84-2aa2503b7ff6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:16.088058 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c393504d-63b7-4729-8c84-2aa2503b7ff6 admin admin] PATH_INFO: `/v3/projects/c61211e2cb1e45db90664e5bc412f2af` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:16.089170 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c393504d-63b7-4729-8c84-2aa2503b7ff6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c61211e2cb1e45db90664e5bc412f2af)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:16.091066 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c393504d-63b7-4729-8c84-2aa2503b7ff6 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:16.176670 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-c393504d-63b7-4729-8c84-2aa2503b7ff6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c61211e2cb1e45db90664e5bc412f2af', 'request_id': 'req-c393504d-63b7-4729-8c84-2aa2503b7ff6'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:16.183524 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-c393504d-63b7-4729-8c84-2aa2503b7ff6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c61211e2cb1e45db90664e5bc412f2af', 'request_id': 'req-c393504d-63b7-4729-8c84-2aa2503b7ff6'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:16.193421 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-c393504d-63b7-4729-8c84-2aa2503b7ff6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c61211e2cb1e45db90664e5bc412f2af', 'request_id': 'req-c393504d-63b7-4729-8c84-2aa2503b7ff6'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:16.295256 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-c393504d-63b7-4729-8c84-2aa2503b7ff6 admin admin] The token cache is being invalidate because project c61211e2cb1e45db90664e5bc412f2af was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:04:16.295551 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-c393504d-63b7-4729-8c84-2aa2503b7ff6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:16.296225 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-c393504d-63b7-4729-8c84-2aa2503b7ff6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:16.297917 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1300/2620] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 24 10:04:16 2022] DELETE /identity/v3/projects/c61211e2cb1e45db90664e5bc412f2af => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:04:16.303125 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f02bd8d-5316-4ca1-b3a8-cc0f5c3221c8 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:16.303623 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f02bd8d-5316-4ca1-b3a8-cc0f5c3221c8 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:16.303728 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f02bd8d-5316-4ca1-b3a8-cc0f5c3221c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:16.323439 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-2f02bd8d-5316-4ca1-b3a8-cc0f5c3221c8 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:04:16.324690 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-2f02bd8d-5316-4ca1-b3a8-cc0f5c3221c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:16.345871 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:04:16.384336 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1321/2621] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 10:04:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:04:16.392299 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6f254e-3bd4-40c0-898a-d59a18c22f64 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:16.392592 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6f254e-3bd4-40c0-898a-d59a18c22f64 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:16.392879 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6f254e-3bd4-40c0-898a-d59a18c22f64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:16.413287 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-da6f254e-3bd4-40c0-898a-d59a18c22f64 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:04:16.414532 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-da6f254e-3bd4-40c0-898a-d59a18c22f64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:16.422662 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1301/2622] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 10:04:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:04:16.427186 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f02bd8d-5316-4ca1-b3a8-cc0f5c3221c8 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:16.437752 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f02bd8d-5316-4ca1-b3a8-cc0f5c3221c8 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:16.441478 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9334a010-b8d7-46cc-b03e-b72a325423fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:16.442079 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9334a010-b8d7-46cc-b03e-b72a325423fa admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:16.442390 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9334a010-b8d7-46cc-b03e-b72a325423fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:16.443123 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9334a010-b8d7-46cc-b03e-b72a325423fa admin admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:16.443330 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9334a010-b8d7-46cc-b03e-b72a325423fa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:04:16.443671 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9334a010-b8d7-46cc-b03e-b72a325423fa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:16.445266 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9334a010-b8d7-46cc-b03e-b72a325423fa admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:16.456788 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1322/2623] 10.222.0.16 () {58 vars in 1153 bytes} [Wed Aug 24 10:04:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:04:16.461478 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6f254e-3bd4-40c0-898a-d59a18c22f64 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:16.467186 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:04:16.473540 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6f254e-3bd4-40c0-898a-d59a18c22f64 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:16.476279 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5a497e-c58b-4370-b99c-6dbbb448180f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:16.476875 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5a497e-c58b-4370-b99c-6dbbb448180f admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:16.477213 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5a497e-c58b-4370-b99c-6dbbb448180f admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:16.477516 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5a497e-c58b-4370-b99c-6dbbb448180f admin admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:16.478144 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5a497e-c58b-4370-b99c-6dbbb448180f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:16.480134 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5a497e-c58b-4370-b99c-6dbbb448180f admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:16.581245 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1302/2624] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 24 10:04:16 2022] POST /identity/v3/projects => generated 387 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:04:16.586073 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9334a010-b8d7-46cc-b03e-b72a325423fa admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:16.595934 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9334a010-b8d7-46cc-b03e-b72a325423fa admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:16.653483 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bb24ff-6e95-47c0-8941-5e053371b318 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:16.654090 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bb24ff-6e95-47c0-8941-5e053371b318 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:16.654372 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bb24ff-6e95-47c0-8941-5e053371b318 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:16.654910 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bb24ff-6e95-47c0-8941-5e053371b318 admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:16.655545 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bb24ff-6e95-47c0-8941-5e053371b318 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:16.657596 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bb24ff-6e95-47c0-8941-5e053371b318 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:16.733709 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1323/2625] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 24 10:04:16 2022] POST /identity/v3/users => generated 294 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:04:16.737822 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5a497e-c58b-4370-b99c-6dbbb448180f admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:16.748069 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5a497e-c58b-4370-b99c-6dbbb448180f admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:16.752672 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9304f2cd-6de8-45a4-8799-daccecd954c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:16.753272 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9304f2cd-6de8-45a4-8799-daccecd954c7 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:16.753551 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9304f2cd-6de8-45a4-8799-daccecd954c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:16.753826 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9304f2cd-6de8-45a4-8799-daccecd954c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:16.754474 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9304f2cd-6de8-45a4-8799-daccecd954c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:16.756667 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9304f2cd-6de8-45a4-8799-daccecd954c7 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:16.766532 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1303/2626] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 24 10:04:16 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:04:16.779837 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bb24ff-6e95-47c0-8941-5e053371b318 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:16.789900 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bb24ff-6e95-47c0-8941-5e053371b318 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:16.798053 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24b0b38-e8ae-4691-9a2c-ab577b257764 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:16.798696 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24b0b38-e8ae-4691-9a2c-ab577b257764 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:16.798920 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24b0b38-e8ae-4691-9a2c-ab577b257764 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:16.799193 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24b0b38-e8ae-4691-9a2c-ab577b257764 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:16.799864 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24b0b38-e8ae-4691-9a2c-ab577b257764 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:16.801436 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24b0b38-e8ae-4691-9a2c-ab577b257764 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:16.812989 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1324/2627] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 24 10:04:16 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:04:16.816989 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9304f2cd-6de8-45a4-8799-daccecd954c7 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:16.827324 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9304f2cd-6de8-45a4-8799-daccecd954c7 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:16.832186 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0500e0db-ea88-405e-800f-3533d179cd4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:16.832837 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0500e0db-ea88-405e-800f-3533d179cd4a admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:16.833116 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0500e0db-ea88-405e-800f-3533d179cd4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:16.833391 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0500e0db-ea88-405e-800f-3533d179cd4a admin admin] PATH_INFO: `/v3/projects/d12e0c3b51dc4d04a9bed6716946fe32/users/129ac015c6264a7a939a6da9992fab54/roles/136cdf78e00a426997dd4e37faff489b` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:16.849082 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0500e0db-ea88-405e-800f-3533d179cd4a admin admin] RBAC: Authorizing `identity:create_grant(project_id=d12e0c3b51dc4d04a9bed6716946fe32, user_id=129ac015c6264a7a939a6da9992fab54, role_id=136cdf78e00a426997dd4e37faff489b)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:16.851776 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0500e0db-ea88-405e-800f-3533d179cd4a admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:16.993213 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1304/2628] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 24 10:04:16 2022] PUT /identity/v3/projects/d12e0c3b51dc4d04a9bed6716946fe32/users/129ac015c6264a7a939a6da9992fab54/roles/136cdf78e00a426997dd4e37faff489b => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:04:17.003592 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a9195a-ad92-4a99-9407-6b99a9b80900 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:17.003891 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a9195a-ad92-4a99-9407-6b99a9b80900 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:17.004165 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a9195a-ad92-4a99-9407-6b99a9b80900 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:17.033065 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-68a9195a-ad92-4a99-9407-6b99a9b80900 None None] MFA Rules not processed for user `129ac015c6264a7a939a6da9992fab54`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:04:17.034315 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-68a9195a-ad92-4a99-9407-6b99a9b80900 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:17.101373 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1325/2629] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 10:04:17 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:04:17.105970 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0500e0db-ea88-405e-800f-3533d179cd4a admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:17.167403 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0500e0db-ea88-405e-800f-3533d179cd4a admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:17.172148 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7927b419-0fff-4284-be52-26507449a739 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:17.172754 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7927b419-0fff-4284-be52-26507449a739 admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:17.173037 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7927b419-0fff-4284-be52-26507449a739 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:17.173308 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7927b419-0fff-4284-be52-26507449a739 admin admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:17.173941 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7927b419-0fff-4284-be52-26507449a739 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:17.175898 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7927b419-0fff-4284-be52-26507449a739 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:17.263087 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1305/2630] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 24 10:04:17 2022] POST /identity/v3/projects => generated 389 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 10:04:17.267032 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a9195a-ad92-4a99-9407-6b99a9b80900 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:17.277853 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a9195a-ad92-4a99-9407-6b99a9b80900 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:17.295703 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:04:17.338022 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eeda6de-015c-4528-afb9-f8ab00581f2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:17.338695 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eeda6de-015c-4528-afb9-f8ab00581f2f admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:17.338903 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eeda6de-015c-4528-afb9-f8ab00581f2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:17.339174 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eeda6de-015c-4528-afb9-f8ab00581f2f admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:17.339847 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eeda6de-015c-4528-afb9-f8ab00581f2f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:17.341978 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eeda6de-015c-4528-afb9-f8ab00581f2f admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:17.453698 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1326/2631] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 24 10:04:17 2022] POST /identity/v3/users => generated 301 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:04:17.455243 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7927b419-0fff-4284-be52-26507449a739 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:17.467213 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7927b419-0fff-4284-be52-26507449a739 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:17.472092 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa8d431-9a1f-456f-8ac3-dca9d3a371d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:17.472865 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa8d431-9a1f-456f-8ac3-dca9d3a371d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:17.473149 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa8d431-9a1f-456f-8ac3-dca9d3a371d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:17.473424 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa8d431-9a1f-456f-8ac3-dca9d3a371d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:17.474578 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa8d431-9a1f-456f-8ac3-dca9d3a371d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:17.476282 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa8d431-9a1f-456f-8ac3-dca9d3a371d7 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:17.486849 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1306/2632] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 24 10:04:17 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 10:04:17.491562 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eeda6de-015c-4528-afb9-f8ab00581f2f admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:17.501475 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eeda6de-015c-4528-afb9-f8ab00581f2f admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:17.506890 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382ade19-aa13-414b-a461-a031e1639342 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:17.507535 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382ade19-aa13-414b-a461-a031e1639342 admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:17.507846 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382ade19-aa13-414b-a461-a031e1639342 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:17.508117 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382ade19-aa13-414b-a461-a031e1639342 admin admin] PATH_INFO: `/v3/projects/d061a51cb0f54fd98b8a7f51e170cbe5/users/f15743d2a0ce48d7b31377230d1d1319/roles/15f634d4a2d247ddab90c458e5e1c141` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:17.522313 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382ade19-aa13-414b-a461-a031e1639342 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d061a51cb0f54fd98b8a7f51e170cbe5, user_id=f15743d2a0ce48d7b31377230d1d1319, role_id=15f634d4a2d247ddab90c458e5e1c141)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:17.524979 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382ade19-aa13-414b-a461-a031e1639342 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:17.604707 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1327/2633] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 24 10:04:17 2022] PUT /identity/v3/projects/d061a51cb0f54fd98b8a7f51e170cbe5/users/f15743d2a0ce48d7b31377230d1d1319/roles/15f634d4a2d247ddab90c458e5e1c141 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:04:17.608607 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa8d431-9a1f-456f-8ac3-dca9d3a371d7 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:17.619134 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa8d431-9a1f-456f-8ac3-dca9d3a371d7 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:17.673913 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6e260b-a46f-435d-a559-a5eeb541cf15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:17.674451 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6e260b-a46f-435d-a559-a5eeb541cf15 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:17.674763 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6e260b-a46f-435d-a559-a5eeb541cf15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:17.675038 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6e260b-a46f-435d-a559-a5eeb541cf15 admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:17.675713 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6e260b-a46f-435d-a559-a5eeb541cf15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:17.677991 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6e260b-a46f-435d-a559-a5eeb541cf15 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:17.688002 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1307/2634] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 24 10:04:17 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:04:17.692114 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382ade19-aa13-414b-a461-a031e1639342 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:17.702885 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382ade19-aa13-414b-a461-a031e1639342 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:17.707748 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b96cdfb-19f2-4ec7-8720-c92e3c63a563 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:17.708423 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b96cdfb-19f2-4ec7-8720-c92e3c63a563 admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:17.708729 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b96cdfb-19f2-4ec7-8720-c92e3c63a563 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:17.709003 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b96cdfb-19f2-4ec7-8720-c92e3c63a563 admin admin] PATH_INFO: `/v3/projects/d061a51cb0f54fd98b8a7f51e170cbe5/users/f15743d2a0ce48d7b31377230d1d1319/roles/15f634d4a2d247ddab90c458e5e1c141` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:17.710892 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b96cdfb-19f2-4ec7-8720-c92e3c63a563 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d061a51cb0f54fd98b8a7f51e170cbe5, user_id=f15743d2a0ce48d7b31377230d1d1319, role_id=15f634d4a2d247ddab90c458e5e1c141)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:17.713550 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b96cdfb-19f2-4ec7-8720-c92e3c63a563 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:17.723384 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1328/2635] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 24 10:04:17 2022] PUT /identity/v3/projects/d061a51cb0f54fd98b8a7f51e170cbe5/users/f15743d2a0ce48d7b31377230d1d1319/roles/15f634d4a2d247ddab90c458e5e1c141 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:04:17.734911 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdff28b-9cc4-4c66-857a-ca180a16137d None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:17.735201 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdff28b-9cc4-4c66-857a-ca180a16137d None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:17.735481 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdff28b-9cc4-4c66-857a-ca180a16137d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:17.763999 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-5fdff28b-9cc4-4c66-857a-ca180a16137d None None] MFA Rules not processed for user `f15743d2a0ce48d7b31377230d1d1319`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:04:17.765207 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-5fdff28b-9cc4-4c66-857a-ca180a16137d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:17.770089 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:04:17.832794 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1308/2636] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 10:04:17 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:04:17.843697 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b96cdfb-19f2-4ec7-8720-c92e3c63a563 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:17.845385 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-1b96cdfb-19f2-4ec7-8720-c92e3c63a563 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:17.902731 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b96cdfb-19f2-4ec7-8720-c92e3c63a563 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:17.906540 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4585df79-4159-4a95-a0c1-4c3c26179001 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c32b83577c46488906fe409a790b24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:17.907145 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4585df79-4159-4a95-a0c1-4c3c26179001 service cinder] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:17.907424 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4585df79-4159-4a95-a0c1-4c3c26179001 service cinder] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:17.907753 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4585df79-4159-4a95-a0c1-4c3c26179001 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:17.917625 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4585df79-4159-4a95-a0c1-4c3c26179001 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:17.919051 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4585df79-4159-4a95-a0c1-4c3c26179001 service cinder] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:17.928511 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1329/2637] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 10:04:17 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:04:18.063483 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdff28b-9cc4-4c66-857a-ca180a16137d None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:18.127343 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdff28b-9cc4-4c66-857a-ca180a16137d None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:18.132771 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1960cd8e-da97-4160-897b-1444445a3102 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:18.133403 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1960cd8e-da97-4160-897b-1444445a3102 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:18.133678 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1960cd8e-da97-4160-897b-1444445a3102 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:18.133946 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1960cd8e-da97-4160-897b-1444445a3102 admin admin] PATH_INFO: `/v3/users/129ac015c6264a7a939a6da9992fab54` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:18.135050 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1960cd8e-da97-4160-897b-1444445a3102 admin admin] RBAC: Authorizing `identity:delete_user(user_id=129ac015c6264a7a939a6da9992fab54)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:18.136978 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1960cd8e-da97-4160-897b-1444445a3102 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:18.429272 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-1960cd8e-da97-4160-897b-1444445a3102 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '129ac015c6264a7a939a6da9992fab54'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:18.429927 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-1960cd8e-da97-4160-897b-1444445a3102 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '129ac015c6264a7a939a6da9992fab54'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:18.442051 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-1960cd8e-da97-4160-897b-1444445a3102 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '129ac015c6264a7a939a6da9992fab54'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:18.451274 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-1960cd8e-da97-4160-897b-1444445a3102 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '129ac015c6264a7a939a6da9992fab54'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:18.518454 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-1960cd8e-da97-4160-897b-1444445a3102 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '129ac015c6264a7a939a6da9992fab54'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:18.533023 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1309/2638] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 24 10:04:18 2022] DELETE /identity/v3/users/129ac015c6264a7a939a6da9992fab54 => generated 0 bytes in 470 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:04:18.538292 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f281dd3-c3c9-438e-baa8-2e03e198c74f None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:18.538769 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f281dd3-c3c9-438e-baa8-2e03e198c74f None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:18.538878 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f281dd3-c3c9-438e-baa8-2e03e198c74f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:18.558699 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-0f281dd3-c3c9-438e-baa8-2e03e198c74f None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:04:18.560074 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-0f281dd3-c3c9-438e-baa8-2e03e198c74f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:18.638372 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1330/2639] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 10:04:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:04:18.645534 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1960cd8e-da97-4160-897b-1444445a3102 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:18.647493 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-1960cd8e-da97-4160-897b-1444445a3102 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:18.674809 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:04:18.710862 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1960cd8e-da97-4160-897b-1444445a3102 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:18.716242 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ade504-1440-49fd-bc94-404ebec22ed4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e5260e798404e8bb12b25ca8ea97052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:18.717254 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ade504-1440-49fd-bc94-404ebec22ed4 service neutron] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:18.717535 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ade504-1440-49fd-bc94-404ebec22ed4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:18.717802 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ade504-1440-49fd-bc94-404ebec22ed4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:18.731057 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ade504-1440-49fd-bc94-404ebec22ed4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:18.732525 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ade504-1440-49fd-bc94-404ebec22ed4 service neutron] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:18.742577 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1310/2640] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 10:04:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:04:19.169864 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f281dd3-c3c9-438e-baa8-2e03e198c74f None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:19.171316 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-0f281dd3-c3c9-438e-baa8-2e03e198c74f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:19.184973 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f281dd3-c3c9-438e-baa8-2e03e198c74f None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:19.190031 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d4d218-53b2-4ce2-bd8f-ed64beb7fe20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:19.190681 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d4d218-53b2-4ce2-bd8f-ed64beb7fe20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:19.190968 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d4d218-53b2-4ce2-bd8f-ed64beb7fe20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:19.191247 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d4d218-53b2-4ce2-bd8f-ed64beb7fe20 admin admin] PATH_INFO: `/v3/projects/d12e0c3b51dc4d04a9bed6716946fe32` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:19.192491 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d4d218-53b2-4ce2-bd8f-ed64beb7fe20 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d12e0c3b51dc4d04a9bed6716946fe32)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:19.194930 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d4d218-53b2-4ce2-bd8f-ed64beb7fe20 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:19.207434 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:04:19.313997 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-62d4d218-53b2-4ce2-bd8f-ed64beb7fe20 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd12e0c3b51dc4d04a9bed6716946fe32', 'request_id': 'req-62d4d218-53b2-4ce2-bd8f-ed64beb7fe20'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:19.320900 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-62d4d218-53b2-4ce2-bd8f-ed64beb7fe20 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd12e0c3b51dc4d04a9bed6716946fe32', 'request_id': 'req-62d4d218-53b2-4ce2-bd8f-ed64beb7fe20'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:19.326537 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-62d4d218-53b2-4ce2-bd8f-ed64beb7fe20 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd12e0c3b51dc4d04a9bed6716946fe32', 'request_id': 'req-62d4d218-53b2-4ce2-bd8f-ed64beb7fe20'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:19.403199 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-62d4d218-53b2-4ce2-bd8f-ed64beb7fe20 admin admin] The token cache is being invalidate because project d12e0c3b51dc4d04a9bed6716946fe32 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81602) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:04:19.403487 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-62d4d218-53b2-4ce2-bd8f-ed64beb7fe20 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:19.404119 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-62d4d218-53b2-4ce2-bd8f-ed64beb7fe20 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:19.405587 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1331/2641] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 24 10:04:19 2022] DELETE /identity/v3/projects/d12e0c3b51dc4d04a9bed6716946fe32 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:04:19.409735 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ade504-1440-49fd-bc94-404ebec22ed4 service neutron] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:19.421454 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ade504-1440-49fd-bc94-404ebec22ed4 service neutron] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:19.477742 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c96f50-044f-4bb3-b116-43e2071f0728 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:19.477742 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c96f50-044f-4bb3-b116-43e2071f0728 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:19.478062 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c96f50-044f-4bb3-b116-43e2071f0728 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:19.478062 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c96f50-044f-4bb3-b116-43e2071f0728 admin admin] PATH_INFO: `/v3/users/f15743d2a0ce48d7b31377230d1d1319` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:19.479165 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c96f50-044f-4bb3-b116-43e2071f0728 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f15743d2a0ce48d7b31377230d1d1319)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:19.481107 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c96f50-044f-4bb3-b116-43e2071f0728 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:19.705175 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-32c96f50-044f-4bb3-b116-43e2071f0728 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f15743d2a0ce48d7b31377230d1d1319'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:19.705952 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-32c96f50-044f-4bb3-b116-43e2071f0728 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f15743d2a0ce48d7b31377230d1d1319'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:19.718586 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-32c96f50-044f-4bb3-b116-43e2071f0728 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f15743d2a0ce48d7b31377230d1d1319'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:19.726316 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-32c96f50-044f-4bb3-b116-43e2071f0728 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f15743d2a0ce48d7b31377230d1d1319'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:19.796593 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-32c96f50-044f-4bb3-b116-43e2071f0728 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f15743d2a0ce48d7b31377230d1d1319'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:19.803161 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1311/2642] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 24 10:04:19 2022] DELETE /identity/v3/users/f15743d2a0ce48d7b31377230d1d1319 => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:04:20.222461 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d4d218-53b2-4ce2-bd8f-ed64beb7fe20 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:20.223994 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-62d4d218-53b2-4ce2-bd8f-ed64beb7fe20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:20.237967 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d4d218-53b2-4ce2-bd8f-ed64beb7fe20 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:20.295908 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c8ff55-58ea-47c5-bcca-98f5d9d46c04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:20.296550 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c8ff55-58ea-47c5-bcca-98f5d9d46c04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:20.296834 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c8ff55-58ea-47c5-bcca-98f5d9d46c04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:20.297118 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c8ff55-58ea-47c5-bcca-98f5d9d46c04 admin admin] PATH_INFO: `/v3/projects/d061a51cb0f54fd98b8a7f51e170cbe5` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:20.298247 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c8ff55-58ea-47c5-bcca-98f5d9d46c04 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d061a51cb0f54fd98b8a7f51e170cbe5)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:20.300239 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c8ff55-58ea-47c5-bcca-98f5d9d46c04 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:20.393148 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-b6c8ff55-58ea-47c5-bcca-98f5d9d46c04 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd061a51cb0f54fd98b8a7f51e170cbe5', 'request_id': 'req-b6c8ff55-58ea-47c5-bcca-98f5d9d46c04'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:20.400278 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-b6c8ff55-58ea-47c5-bcca-98f5d9d46c04 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd061a51cb0f54fd98b8a7f51e170cbe5', 'request_id': 'req-b6c8ff55-58ea-47c5-bcca-98f5d9d46c04'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:20.405955 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-b6c8ff55-58ea-47c5-bcca-98f5d9d46c04 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd061a51cb0f54fd98b8a7f51e170cbe5', 'request_id': 'req-b6c8ff55-58ea-47c5-bcca-98f5d9d46c04'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:20.483740 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-b6c8ff55-58ea-47c5-bcca-98f5d9d46c04 admin admin] The token cache is being invalidate because project d061a51cb0f54fd98b8a7f51e170cbe5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81602) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:04:20.483953 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-b6c8ff55-58ea-47c5-bcca-98f5d9d46c04 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:20.484564 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-b6c8ff55-58ea-47c5-bcca-98f5d9d46c04 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:20.486090 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1332/2643] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 24 10:04:20 2022] DELETE /identity/v3/projects/d061a51cb0f54fd98b8a7f51e170cbe5 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:04:20.491152 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dcccd9-eb8e-412c-9773-8d05243010be None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:20.491649 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dcccd9-eb8e-412c-9773-8d05243010be None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:20.491743 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dcccd9-eb8e-412c-9773-8d05243010be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:20.511509 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-04dcccd9-eb8e-412c-9773-8d05243010be None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:04:20.512725 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-04dcccd9-eb8e-412c-9773-8d05243010be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:20.552596 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:04:20.571634 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1312/2644] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 10:04:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:04:20.579339 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023d31b5-0a3f-4d6d-8483-d43f21a9ffd8 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:20.579659 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023d31b5-0a3f-4d6d-8483-d43f21a9ffd8 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:20.580203 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023d31b5-0a3f-4d6d-8483-d43f21a9ffd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:20.601192 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-023d31b5-0a3f-4d6d-8483-d43f21a9ffd8 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:04:20.602409 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-023d31b5-0a3f-4d6d-8483-d43f21a9ffd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:20.610384 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1333/2645] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 10:04:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:04:20.614632 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dcccd9-eb8e-412c-9773-8d05243010be None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:20.624494 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dcccd9-eb8e-412c-9773-8d05243010be None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:20.629105 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5980a973-4abc-4fce-a2ab-701a0d34c6cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:20.636033 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5980a973-4abc-4fce-a2ab-701a0d34c6cf admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:20.636033 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5980a973-4abc-4fce-a2ab-701a0d34c6cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:20.636033 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5980a973-4abc-4fce-a2ab-701a0d34c6cf admin admin] PATH_INFO: `/v3/domains` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:20.636033 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5980a973-4abc-4fce-a2ab-701a0d34c6cf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:04:20.636033 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5980a973-4abc-4fce-a2ab-701a0d34c6cf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:20.636033 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5980a973-4abc-4fce-a2ab-701a0d34c6cf admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:20.645035 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1313/2646] 10.222.0.16 () {58 vars in 1153 bytes} [Wed Aug 24 10:04:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:04:20.649851 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023d31b5-0a3f-4d6d-8483-d43f21a9ffd8 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:20.659115 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023d31b5-0a3f-4d6d-8483-d43f21a9ffd8 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:20.663853 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ee818b-1ba8-406e-a7ae-e4ec6acf16c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:20.664449 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ee818b-1ba8-406e-a7ae-e4ec6acf16c7 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:20.664730 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ee818b-1ba8-406e-a7ae-e4ec6acf16c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:20.665003 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ee818b-1ba8-406e-a7ae-e4ec6acf16c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:20.665644 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ee818b-1ba8-406e-a7ae-e4ec6acf16c7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:20.667550 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ee818b-1ba8-406e-a7ae-e4ec6acf16c7 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:20.750174 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1334/2647] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 24 10:04:20 2022] POST /identity/v3/projects => generated 387 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:04:20.754446 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5980a973-4abc-4fce-a2ab-701a0d34c6cf admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:20.765358 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5980a973-4abc-4fce-a2ab-701a0d34c6cf admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:20.820708 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0560524a-45f8-4bb2-a3a4-5cbff094ec56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:20.821322 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0560524a-45f8-4bb2-a3a4-5cbff094ec56 admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:20.821607 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0560524a-45f8-4bb2-a3a4-5cbff094ec56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:20.821878 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0560524a-45f8-4bb2-a3a4-5cbff094ec56 admin admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:20.822497 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0560524a-45f8-4bb2-a3a4-5cbff094ec56 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:20.826800 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0560524a-45f8-4bb2-a3a4-5cbff094ec56 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:20.941825 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1314/2648] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 24 10:04:20 2022] POST /identity/v3/users => generated 294 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:04:20.946663 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ee818b-1ba8-406e-a7ae-e4ec6acf16c7 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:20.952290 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:04:20.957821 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ee818b-1ba8-406e-a7ae-e4ec6acf16c7 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:20.962861 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf40d792-29e1-45cd-9357-f01dc1e548ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:20.963468 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf40d792-29e1-45cd-9357-f01dc1e548ce admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:20.963786 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf40d792-29e1-45cd-9357-f01dc1e548ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:20.964064 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf40d792-29e1-45cd-9357-f01dc1e548ce admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:20.964714 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf40d792-29e1-45cd-9357-f01dc1e548ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:20.966299 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf40d792-29e1-45cd-9357-f01dc1e548ce admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:20.977485 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1335/2649] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 24 10:04:20 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 10:04:20.984694 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0560524a-45f8-4bb2-a3a4-5cbff094ec56 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:20.995438 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0560524a-45f8-4bb2-a3a4-5cbff094ec56 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:21.000134 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10336535-c406-4272-b192-1627cbf5d5c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:21.007498 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10336535-c406-4272-b192-1627cbf5d5c0 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:21.007498 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10336535-c406-4272-b192-1627cbf5d5c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:21.007498 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10336535-c406-4272-b192-1627cbf5d5c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:21.007498 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10336535-c406-4272-b192-1627cbf5d5c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:21.007498 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10336535-c406-4272-b192-1627cbf5d5c0 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:21.015063 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1315/2650] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 24 10:04:20 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:04:21.019289 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf40d792-29e1-45cd-9357-f01dc1e548ce admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:21.030031 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf40d792-29e1-45cd-9357-f01dc1e548ce admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:21.035419 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7d23de-dc0d-44c9-bec8-813d6767a188 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:21.036111 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7d23de-dc0d-44c9-bec8-813d6767a188 admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:21.036393 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7d23de-dc0d-44c9-bec8-813d6767a188 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:21.041053 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7d23de-dc0d-44c9-bec8-813d6767a188 admin admin] PATH_INFO: `/v3/projects/e515d57acce0487c8c095a54c38e250f/users/83d6e2076fcb43e4bd94f608579d94e7/roles/136cdf78e00a426997dd4e37faff489b` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:21.053313 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7d23de-dc0d-44c9-bec8-813d6767a188 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e515d57acce0487c8c095a54c38e250f, user_id=83d6e2076fcb43e4bd94f608579d94e7, role_id=136cdf78e00a426997dd4e37faff489b)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:21.056014 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7d23de-dc0d-44c9-bec8-813d6767a188 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:21.118492 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1336/2651] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 24 10:04:21 2022] PUT /identity/v3/projects/e515d57acce0487c8c095a54c38e250f/users/83d6e2076fcb43e4bd94f608579d94e7/roles/136cdf78e00a426997dd4e37faff489b => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:04:21.128749 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec172ac-e23a-4e6d-a722-4d3ca843a39f None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:21.129032 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec172ac-e23a-4e6d-a722-4d3ca843a39f None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:21.129306 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec172ac-e23a-4e6d-a722-4d3ca843a39f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:21.156985 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-2ec172ac-e23a-4e6d-a722-4d3ca843a39f None None] MFA Rules not processed for user `83d6e2076fcb43e4bd94f608579d94e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:04:21.158200 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-2ec172ac-e23a-4e6d-a722-4d3ca843a39f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:21.222648 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1316/2652] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 10:04:21 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:04:21.227808 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7d23de-dc0d-44c9-bec8-813d6767a188 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:21.293448 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7d23de-dc0d-44c9-bec8-813d6767a188 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:21.298163 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc70f7c-45bc-439b-b54b-a1995593bbdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:21.298768 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc70f7c-45bc-439b-b54b-a1995593bbdc admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:21.299046 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc70f7c-45bc-439b-b54b-a1995593bbdc admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:21.299317 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc70f7c-45bc-439b-b54b-a1995593bbdc admin admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:21.300058 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc70f7c-45bc-439b-b54b-a1995593bbdc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:21.301959 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc70f7c-45bc-439b-b54b-a1995593bbdc admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:21.406754 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1337/2653] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 24 10:04:21 2022] POST /identity/v3/projects => generated 387 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:04:21.410879 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec172ac-e23a-4e6d-a722-4d3ca843a39f None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:21.422284 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec172ac-e23a-4e6d-a722-4d3ca843a39f None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:21.460568 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:04:21.478363 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbfef2a-0a22-4fe3-8b04-e33c4215a958 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:21.478975 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbfef2a-0a22-4fe3-8b04-e33c4215a958 admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:21.479254 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbfef2a-0a22-4fe3-8b04-e33c4215a958 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:21.479529 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbfef2a-0a22-4fe3-8b04-e33c4215a958 admin admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:21.480181 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbfef2a-0a22-4fe3-8b04-e33c4215a958 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:21.482119 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbfef2a-0a22-4fe3-8b04-e33c4215a958 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:21.559425 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1317/2654] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 24 10:04:21 2022] POST /identity/v3/users => generated 300 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:04:21.563513 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc70f7c-45bc-439b-b54b-a1995593bbdc admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:21.574066 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc70f7c-45bc-439b-b54b-a1995593bbdc admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:21.578670 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06df9d5-45b0-442f-aecc-3ceb9eff1914 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:21.579268 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06df9d5-45b0-442f-aecc-3ceb9eff1914 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:21.579549 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06df9d5-45b0-442f-aecc-3ceb9eff1914 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:21.579854 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06df9d5-45b0-442f-aecc-3ceb9eff1914 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:21.580759 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06df9d5-45b0-442f-aecc-3ceb9eff1914 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:21.582343 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06df9d5-45b0-442f-aecc-3ceb9eff1914 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:21.593070 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1338/2655] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 24 10:04:21 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:04:21.597094 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbfef2a-0a22-4fe3-8b04-e33c4215a958 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:21.607464 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbfef2a-0a22-4fe3-8b04-e33c4215a958 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:21.612058 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b8a08b-2c53-4e64-8137-85d8a8c09ad6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:21.612702 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b8a08b-2c53-4e64-8137-85d8a8c09ad6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:21.612981 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b8a08b-2c53-4e64-8137-85d8a8c09ad6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:21.613258 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b8a08b-2c53-4e64-8137-85d8a8c09ad6 admin admin] PATH_INFO: `/v3/projects/07ba57694f4744e0b50f8d429ee2df1b/users/3f8c2ca6ea0e4dd0a365993afe08162a/roles/15f634d4a2d247ddab90c458e5e1c141` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:21.627785 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b8a08b-2c53-4e64-8137-85d8a8c09ad6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=07ba57694f4744e0b50f8d429ee2df1b, user_id=3f8c2ca6ea0e4dd0a365993afe08162a, role_id=15f634d4a2d247ddab90c458e5e1c141)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:21.630424 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b8a08b-2c53-4e64-8137-85d8a8c09ad6 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:21.702270 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1318/2656] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 24 10:04:21 2022] PUT /identity/v3/projects/07ba57694f4744e0b50f8d429ee2df1b/users/3f8c2ca6ea0e4dd0a365993afe08162a/roles/15f634d4a2d247ddab90c458e5e1c141 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 10:04:21.706589 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06df9d5-45b0-442f-aecc-3ceb9eff1914 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:21.717012 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06df9d5-45b0-442f-aecc-3ceb9eff1914 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:21.779700 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe64b6ae-a46a-4485-b8ec-63a9cc94046d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:21.780320 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe64b6ae-a46a-4485-b8ec-63a9cc94046d admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:21.780603 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe64b6ae-a46a-4485-b8ec-63a9cc94046d admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:21.780900 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe64b6ae-a46a-4485-b8ec-63a9cc94046d admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:21.781654 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe64b6ae-a46a-4485-b8ec-63a9cc94046d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:21.783279 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe64b6ae-a46a-4485-b8ec-63a9cc94046d admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:21.794669 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1339/2657] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 24 10:04:21 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:04:21.798936 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b8a08b-2c53-4e64-8137-85d8a8c09ad6 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:21.809195 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b8a08b-2c53-4e64-8137-85d8a8c09ad6 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:21.817072 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1a6e0c-d060-4159-a6a3-dfd07fc1b860 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:21.817778 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1a6e0c-d060-4159-a6a3-dfd07fc1b860 admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:21.818035 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1a6e0c-d060-4159-a6a3-dfd07fc1b860 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:21.818305 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1a6e0c-d060-4159-a6a3-dfd07fc1b860 admin admin] PATH_INFO: `/v3/projects/07ba57694f4744e0b50f8d429ee2df1b/users/3f8c2ca6ea0e4dd0a365993afe08162a/roles/15f634d4a2d247ddab90c458e5e1c141` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:21.820341 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1a6e0c-d060-4159-a6a3-dfd07fc1b860 admin admin] RBAC: Authorizing `identity:create_grant(project_id=07ba57694f4744e0b50f8d429ee2df1b, user_id=3f8c2ca6ea0e4dd0a365993afe08162a, role_id=15f634d4a2d247ddab90c458e5e1c141)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:21.828213 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1a6e0c-d060-4159-a6a3-dfd07fc1b860 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:21.835598 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1319/2658] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 24 10:04:21 2022] PUT /identity/v3/projects/07ba57694f4744e0b50f8d429ee2df1b/users/3f8c2ca6ea0e4dd0a365993afe08162a/roles/15f634d4a2d247ddab90c458e5e1c141 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:04:21.846809 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670167c8-51da-44cd-bc9d-d2de45c4b81c None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:21.847103 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670167c8-51da-44cd-bc9d-d2de45c4b81c None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:21.847384 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670167c8-51da-44cd-bc9d-d2de45c4b81c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:21.877175 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-670167c8-51da-44cd-bc9d-d2de45c4b81c None None] MFA Rules not processed for user `3f8c2ca6ea0e4dd0a365993afe08162a`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:04:21.878398 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-670167c8-51da-44cd-bc9d-d2de45c4b81c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:21.906211 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:04:21.948694 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1340/2659] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 10:04:21 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:04:21.960294 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1a6e0c-d060-4159-a6a3-dfd07fc1b860 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:21.962101 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-0c1a6e0c-d060-4159-a6a3-dfd07fc1b860 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:22.025165 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1a6e0c-d060-4159-a6a3-dfd07fc1b860 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:22.029719 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604409b5-ef1c-4646-bedb-0523a1577c13 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c32b83577c46488906fe409a790b24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:22.030500 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604409b5-ef1c-4646-bedb-0523a1577c13 service cinder] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:22.030806 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604409b5-ef1c-4646-bedb-0523a1577c13 service cinder] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:22.031083 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604409b5-ef1c-4646-bedb-0523a1577c13 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:22.041832 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604409b5-ef1c-4646-bedb-0523a1577c13 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:22.043348 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604409b5-ef1c-4646-bedb-0523a1577c13 service cinder] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:22.054151 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1320/2660] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 10:04:21 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:04:23.035254 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670167c8-51da-44cd-bc9d-d2de45c4b81c None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:23.037869 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-670167c8-51da-44cd-bc9d-d2de45c4b81c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:23.067855 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604409b5-ef1c-4646-bedb-0523a1577c13 service cinder] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:23.070020 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-604409b5-ef1c-4646-bedb-0523a1577c13 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:23.124982 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670167c8-51da-44cd-bc9d-d2de45c4b81c None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:23.131331 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5b7ae1-f174-474d-810b-57d0f34ce82b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:23.132024 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5b7ae1-f174-474d-810b-57d0f34ce82b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:23.132322 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5b7ae1-f174-474d-810b-57d0f34ce82b admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:23.132815 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5b7ae1-f174-474d-810b-57d0f34ce82b admin admin] PATH_INFO: `/v3/users/3f8b99f006e84eefa53dc254c07c4418` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:23.133792 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5b7ae1-f174-474d-810b-57d0f34ce82b admin admin] RBAC: Authorizing `identity:delete_user(user_id=3f8b99f006e84eefa53dc254c07c4418)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:23.135764 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5b7ae1-f174-474d-810b-57d0f34ce82b admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:23.181418 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604409b5-ef1c-4646-bedb-0523a1577c13 service cinder] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:23.187340 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcb9228-f9c2-4811-b207-2e8bc1d6eca0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f055f7dc51d43bda9b315ca67f11d58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:23.189152 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcb9228-f9c2-4811-b207-2e8bc1d6eca0 service nova] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:23.189152 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcb9228-f9c2-4811-b207-2e8bc1d6eca0 service nova] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:23.189152 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcb9228-f9c2-4811-b207-2e8bc1d6eca0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:23.192350 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-0bcb9228-f9c2-4811-b207-2e8bc1d6eca0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:23.204090 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcb9228-f9c2-4811-b207-2e8bc1d6eca0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:23.206142 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcb9228-f9c2-4811-b207-2e8bc1d6eca0 service nova] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:23.218085 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1321/2661] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 10:04:23 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 10:04:23.245189 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcb9228-f9c2-4811-b207-2e8bc1d6eca0 service nova] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:23.257498 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcb9228-f9c2-4811-b207-2e8bc1d6eca0 service nova] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:23.261966 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd284c50-f391-4b8b-8e52-0137c059ce1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:23.262611 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd284c50-f391-4b8b-8e52-0137c059ce1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:23.262896 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd284c50-f391-4b8b-8e52-0137c059ce1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:23.263171 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd284c50-f391-4b8b-8e52-0137c059ce1c admin admin] PATH_INFO: `/v3/users/83d6e2076fcb43e4bd94f608579d94e7` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:23.267234 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd284c50-f391-4b8b-8e52-0137c059ce1c admin admin] RBAC: Authorizing `identity:delete_user(user_id=83d6e2076fcb43e4bd94f608579d94e7)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:23.267234 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd284c50-f391-4b8b-8e52-0137c059ce1c admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:23.283627 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:04:23.304677 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-5b5b7ae1-f174-474d-810b-57d0f34ce82b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3f8b99f006e84eefa53dc254c07c4418'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:23.305416 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-5b5b7ae1-f174-474d-810b-57d0f34ce82b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3f8b99f006e84eefa53dc254c07c4418'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:23.381489 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-5b5b7ae1-f174-474d-810b-57d0f34ce82b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3f8b99f006e84eefa53dc254c07c4418'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:23.382705 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-5b5b7ae1-f174-474d-810b-57d0f34ce82b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3f8b99f006e84eefa53dc254c07c4418'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:23.391203 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-5b5b7ae1-f174-474d-810b-57d0f34ce82b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3f8b99f006e84eefa53dc254c07c4418'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:23.409439 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1341/2662] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 24 10:04:23 2022] DELETE /identity/v3/users/3f8b99f006e84eefa53dc254c07c4418 => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:04:23.414515 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbf3465-1be8-426a-aff2-0c8285e195f9 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:23.414804 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbf3465-1be8-426a-aff2-0c8285e195f9 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:23.415110 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbf3465-1be8-426a-aff2-0c8285e195f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:23.434084 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-5dbf3465-1be8-426a-aff2-0c8285e195f9 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:04:23.435349 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-5dbf3465-1be8-426a-aff2-0c8285e195f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:23.465748 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-bd284c50-f391-4b8b-8e52-0137c059ce1c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '83d6e2076fcb43e4bd94f608579d94e7'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:23.466444 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-bd284c50-f391-4b8b-8e52-0137c059ce1c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '83d6e2076fcb43e4bd94f608579d94e7'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:23.476392 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:04:23.480245 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-bd284c50-f391-4b8b-8e52-0137c059ce1c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '83d6e2076fcb43e4bd94f608579d94e7'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:23.488244 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-bd284c50-f391-4b8b-8e52-0137c059ce1c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '83d6e2076fcb43e4bd94f608579d94e7'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:23.501196 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1342/2663] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 10:04:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:04:23.511123 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbf3465-1be8-426a-aff2-0c8285e195f9 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:23.512601 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-5dbf3465-1be8-426a-aff2-0c8285e195f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:23.558406 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-bd284c50-f391-4b8b-8e52-0137c059ce1c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '83d6e2076fcb43e4bd94f608579d94e7'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:23.564944 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1322/2664] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 24 10:04:23 2022] DELETE /identity/v3/users/83d6e2076fcb43e4bd94f608579d94e7 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:04:23.569691 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b626bbb0-aae1-4f64-be0a-a1bf9055d9b8 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:23.570048 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b626bbb0-aae1-4f64-be0a-a1bf9055d9b8 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:23.570404 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b626bbb0-aae1-4f64-be0a-a1bf9055d9b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:23.573924 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbf3465-1be8-426a-aff2-0c8285e195f9 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:23.592178 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-b626bbb0-aae1-4f64-be0a-a1bf9055d9b8 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:04:23.593333 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-b626bbb0-aae1-4f64-be0a-a1bf9055d9b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:23.631783 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3646974-baac-4788-a41f-6b9d9f7278eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e5260e798404e8bb12b25ca8ea97052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:23.632411 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3646974-baac-4788-a41f-6b9d9f7278eb service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:23.632684 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3646974-baac-4788-a41f-6b9d9f7278eb service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:23.632974 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3646974-baac-4788-a41f-6b9d9f7278eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:23.634897 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-f3646974-baac-4788-a41f-6b9d9f7278eb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:23.665219 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1323/2665] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 10:04:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:04:23.675383 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b626bbb0-aae1-4f64-be0a-a1bf9055d9b8 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:23.677028 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-b626bbb0-aae1-4f64-be0a-a1bf9055d9b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:23.690956 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b626bbb0-aae1-4f64-be0a-a1bf9055d9b8 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:23.694728 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b0b8b5-5308-4f06-8dda-e905c39529cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e5260e798404e8bb12b25ca8ea97052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:23.695350 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b0b8b5-5308-4f06-8dda-e905c39529cb service neutron] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:23.695663 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b0b8b5-5308-4f06-8dda-e905c39529cb service neutron] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:23.696068 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b0b8b5-5308-4f06-8dda-e905c39529cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:23.705755 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3646974-baac-4788-a41f-6b9d9f7278eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:23.705755 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3646974-baac-4788-a41f-6b9d9f7278eb service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:23.706118 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b0b8b5-5308-4f06-8dda-e905c39529cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:23.707536 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b0b8b5-5308-4f06-8dda-e905c39529cb service neutron] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:23.713132 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1343/2666] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 10:04:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 10:04:23.717288 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1324/2667] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 10:04:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 10:04:24.245480 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3646974-baac-4788-a41f-6b9d9f7278eb service neutron] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:24.247043 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-f3646974-baac-4788-a41f-6b9d9f7278eb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:24.251973 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b0b8b5-5308-4f06-8dda-e905c39529cb service neutron] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:24.253223 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-07b0b8b5-5308-4f06-8dda-e905c39529cb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:24.264467 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3646974-baac-4788-a41f-6b9d9f7278eb service neutron] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:24.270650 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c327f463-8b21-4967-a025-02cdf00b2fb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:24.274546 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c327f463-8b21-4967-a025-02cdf00b2fb1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:24.274546 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c327f463-8b21-4967-a025-02cdf00b2fb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:24.274546 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c327f463-8b21-4967-a025-02cdf00b2fb1 admin admin] PATH_INFO: `/v3/projects/e515d57acce0487c8c095a54c38e250f` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:24.274546 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c327f463-8b21-4967-a025-02cdf00b2fb1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e515d57acce0487c8c095a54c38e250f)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:24.275479 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b0b8b5-5308-4f06-8dda-e905c39529cb service neutron] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:24.275821 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c327f463-8b21-4967-a025-02cdf00b2fb1 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:24.281283 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43784a3-bacf-4c70-8d34-18d151d30520 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:24.281995 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43784a3-bacf-4c70-8d34-18d151d30520 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:24.282283 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43784a3-bacf-4c70-8d34-18d151d30520 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:24.282563 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43784a3-bacf-4c70-8d34-18d151d30520 admin admin] PATH_INFO: `/v3/projects/2e31467031a54416b979d1099b610f26` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:24.283761 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43784a3-bacf-4c70-8d34-18d151d30520 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e31467031a54416b979d1099b610f26)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:24.287258 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43784a3-bacf-4c70-8d34-18d151d30520 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:24.416305 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-c43784a3-bacf-4c70-8d34-18d151d30520 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e31467031a54416b979d1099b610f26', 'request_id': 'req-c43784a3-bacf-4c70-8d34-18d151d30520'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:24.423028 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-c43784a3-bacf-4c70-8d34-18d151d30520 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e31467031a54416b979d1099b610f26', 'request_id': 'req-c43784a3-bacf-4c70-8d34-18d151d30520'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:24.431247 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-c43784a3-bacf-4c70-8d34-18d151d30520 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e31467031a54416b979d1099b610f26', 'request_id': 'req-c43784a3-bacf-4c70-8d34-18d151d30520'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:24.439067 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:04:24.443379 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-c327f463-8b21-4967-a025-02cdf00b2fb1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e515d57acce0487c8c095a54c38e250f', 'request_id': 'req-c327f463-8b21-4967-a025-02cdf00b2fb1'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:24.451889 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-c327f463-8b21-4967-a025-02cdf00b2fb1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e515d57acce0487c8c095a54c38e250f', 'request_id': 'req-c327f463-8b21-4967-a025-02cdf00b2fb1'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:24.458725 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-c327f463-8b21-4967-a025-02cdf00b2fb1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e515d57acce0487c8c095a54c38e250f', 'request_id': 'req-c327f463-8b21-4967-a025-02cdf00b2fb1'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:24.506470 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-c43784a3-bacf-4c70-8d34-18d151d30520 admin admin] The token cache is being invalidate because project 2e31467031a54416b979d1099b610f26 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:04:24.506794 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-c43784a3-bacf-4c70-8d34-18d151d30520 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:24.507450 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-c43784a3-bacf-4c70-8d34-18d151d30520 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:24.509055 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1325/2668] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 24 10:04:24 2022] DELETE /identity/v3/projects/2e31467031a54416b979d1099b610f26 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:04:24.512902 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43784a3-bacf-4c70-8d34-18d151d30520 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:24.514365 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-c43784a3-bacf-4c70-8d34-18d151d30520 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:24.519450 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:04:24.577454 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43784a3-bacf-4c70-8d34-18d151d30520 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:24.619441 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-c327f463-8b21-4967-a025-02cdf00b2fb1 admin admin] The token cache is being invalidate because project e515d57acce0487c8c095a54c38e250f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81602) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:04:24.619677 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-c327f463-8b21-4967-a025-02cdf00b2fb1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:24.620298 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-c327f463-8b21-4967-a025-02cdf00b2fb1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:24.621835 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1344/2669] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 24 10:04:24 2022] DELETE /identity/v3/projects/e515d57acce0487c8c095a54c38e250f => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:04:24.626126 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c327f463-8b21-4967-a025-02cdf00b2fb1 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:24.628252 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-c327f463-8b21-4967-a025-02cdf00b2fb1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:24.637263 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae618bc-47d9-4efb-bd3a-68379803c219 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:24.637979 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae618bc-47d9-4efb-bd3a-68379803c219 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:24.638325 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae618bc-47d9-4efb-bd3a-68379803c219 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:24.638716 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae618bc-47d9-4efb-bd3a-68379803c219 admin admin] PATH_INFO: `/v3/users/879bf2c89c4c4309a240a51582a78ba6` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:24.639958 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae618bc-47d9-4efb-bd3a-68379803c219 admin admin] RBAC: Authorizing `identity:delete_user(user_id=879bf2c89c4c4309a240a51582a78ba6)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:24.641936 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae618bc-47d9-4efb-bd3a-68379803c219 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:24.692160 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c327f463-8b21-4967-a025-02cdf00b2fb1 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:24.697213 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a142194-90ec-40c7-8896-e606df9cb0a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:24.697842 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a142194-90ec-40c7-8896-e606df9cb0a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:24.698125 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a142194-90ec-40c7-8896-e606df9cb0a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:24.698405 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a142194-90ec-40c7-8896-e606df9cb0a3 admin admin] PATH_INFO: `/v3/users/3f8c2ca6ea0e4dd0a365993afe08162a` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:24.699541 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a142194-90ec-40c7-8896-e606df9cb0a3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3f8c2ca6ea0e4dd0a365993afe08162a)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:24.701487 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a142194-90ec-40c7-8896-e606df9cb0a3 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:24.856339 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-3ae618bc-47d9-4efb-bd3a-68379803c219 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '879bf2c89c4c4309a240a51582a78ba6'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:24.856991 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-3ae618bc-47d9-4efb-bd3a-68379803c219 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '879bf2c89c4c4309a240a51582a78ba6'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:24.869003 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-3ae618bc-47d9-4efb-bd3a-68379803c219 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '879bf2c89c4c4309a240a51582a78ba6'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:24.876084 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-3ae618bc-47d9-4efb-bd3a-68379803c219 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '879bf2c89c4c4309a240a51582a78ba6'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:24.878823 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-2a142194-90ec-40c7-8896-e606df9cb0a3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3f8c2ca6ea0e4dd0a365993afe08162a'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:24.879498 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-2a142194-90ec-40c7-8896-e606df9cb0a3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3f8c2ca6ea0e4dd0a365993afe08162a'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:24.934010 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-2a142194-90ec-40c7-8896-e606df9cb0a3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3f8c2ca6ea0e4dd0a365993afe08162a'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:24.934684 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-2a142194-90ec-40c7-8896-e606df9cb0a3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3f8c2ca6ea0e4dd0a365993afe08162a'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:24.941952 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-2a142194-90ec-40c7-8896-e606df9cb0a3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3f8c2ca6ea0e4dd0a365993afe08162a'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:24.960253 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1345/2670] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 24 10:04:24 2022] DELETE /identity/v3/users/3f8c2ca6ea0e4dd0a365993afe08162a => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 10:04:24.997003 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-3ae618bc-47d9-4efb-bd3a-68379803c219 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '879bf2c89c4c4309a240a51582a78ba6'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:25.003772 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1326/2671] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 24 10:04:24 2022] DELETE /identity/v3/users/879bf2c89c4c4309a240a51582a78ba6 => generated 0 bytes in 491 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 10:04:25.485864 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a142194-90ec-40c7-8896-e606df9cb0a3 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:25.491261 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-2a142194-90ec-40c7-8896-e606df9cb0a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:25.516895 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae618bc-47d9-4efb-bd3a-68379803c219 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:25.518388 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-3ae618bc-47d9-4efb-bd3a-68379803c219 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:25.557243 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:04:25.570619 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a142194-90ec-40c7-8896-e606df9cb0a3 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:25.578731 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:04:25.619261 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae618bc-47d9-4efb-bd3a-68379803c219 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:25.625173 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6df17e-7011-4512-a91e-bad7bacfbc41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:25.626072 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6df17e-7011-4512-a91e-bad7bacfbc41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:25.626409 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6df17e-7011-4512-a91e-bad7bacfbc41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:25.626780 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6df17e-7011-4512-a91e-bad7bacfbc41 admin admin] PATH_INFO: `/v3/projects/5f6f30d09d52449ab0cf34bc9c1b47b1` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:25.628003 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6df17e-7011-4512-a91e-bad7bacfbc41 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5f6f30d09d52449ab0cf34bc9c1b47b1)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:25.630317 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6df17e-7011-4512-a91e-bad7bacfbc41 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:25.637369 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c29105-e7e4-4820-a559-42df194e136b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:25.638010 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c29105-e7e4-4820-a559-42df194e136b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:25.638298 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c29105-e7e4-4820-a559-42df194e136b admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:25.638585 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c29105-e7e4-4820-a559-42df194e136b admin admin] PATH_INFO: `/v3/projects/07ba57694f4744e0b50f8d429ee2df1b` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:25.639764 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c29105-e7e4-4820-a559-42df194e136b admin admin] RBAC: Authorizing `identity:delete_project(project_id=07ba57694f4744e0b50f8d429ee2df1b)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:25.641837 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c29105-e7e4-4820-a559-42df194e136b admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:25.754794 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-bc6df17e-7011-4512-a91e-bad7bacfbc41 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5f6f30d09d52449ab0cf34bc9c1b47b1', 'request_id': 'req-bc6df17e-7011-4512-a91e-bad7bacfbc41'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:25.755973 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-e3c29105-e7e4-4820-a559-42df194e136b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '07ba57694f4744e0b50f8d429ee2df1b', 'request_id': 'req-e3c29105-e7e4-4820-a559-42df194e136b'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:25.761006 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-bc6df17e-7011-4512-a91e-bad7bacfbc41 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5f6f30d09d52449ab0cf34bc9c1b47b1', 'request_id': 'req-bc6df17e-7011-4512-a91e-bad7bacfbc41'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:25.762504 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-e3c29105-e7e4-4820-a559-42df194e136b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '07ba57694f4744e0b50f8d429ee2df1b', 'request_id': 'req-e3c29105-e7e4-4820-a559-42df194e136b'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:25.766593 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-bc6df17e-7011-4512-a91e-bad7bacfbc41 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5f6f30d09d52449ab0cf34bc9c1b47b1', 'request_id': 'req-bc6df17e-7011-4512-a91e-bad7bacfbc41'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:25.768728 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-e3c29105-e7e4-4820-a559-42df194e136b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '07ba57694f4744e0b50f8d429ee2df1b', 'request_id': 'req-e3c29105-e7e4-4820-a559-42df194e136b'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:25.843392 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-bc6df17e-7011-4512-a91e-bad7bacfbc41 admin admin] The token cache is being invalidate because project 5f6f30d09d52449ab0cf34bc9c1b47b1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:04:25.843717 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-bc6df17e-7011-4512-a91e-bad7bacfbc41 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:25.844503 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-bc6df17e-7011-4512-a91e-bad7bacfbc41 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:25.846181 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1327/2672] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 24 10:04:25 2022] DELETE /identity/v3/projects/5f6f30d09d52449ab0cf34bc9c1b47b1 => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 10:04:25.850000 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6df17e-7011-4512-a91e-bad7bacfbc41 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:25.851294 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-bc6df17e-7011-4512-a91e-bad7bacfbc41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:25.917560 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6df17e-7011-4512-a91e-bad7bacfbc41 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:25.922258 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea990299-5dd6-4f9d-8f6f-d1457bd2dbcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:25.923125 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea990299-5dd6-4f9d-8f6f-d1457bd2dbcd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:25.923478 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea990299-5dd6-4f9d-8f6f-d1457bd2dbcd admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:25.923797 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea990299-5dd6-4f9d-8f6f-d1457bd2dbcd admin admin] PATH_INFO: `/v3/users/aad8f8b8a6cf48019b7f987bd5d1cd3c` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:25.924925 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea990299-5dd6-4f9d-8f6f-d1457bd2dbcd admin admin] RBAC: Authorizing `identity:delete_user(user_id=aad8f8b8a6cf48019b7f987bd5d1cd3c)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:25.926815 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea990299-5dd6-4f9d-8f6f-d1457bd2dbcd admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:25.941290 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-e3c29105-e7e4-4820-a559-42df194e136b admin admin] The token cache is being invalidate because project 07ba57694f4744e0b50f8d429ee2df1b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81602) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:04:25.941677 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-e3c29105-e7e4-4820-a559-42df194e136b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:25.942371 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-e3c29105-e7e4-4820-a559-42df194e136b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:25.944513 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1346/2673] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 24 10:04:25 2022] DELETE /identity/v3/projects/07ba57694f4744e0b50f8d429ee2df1b => generated 0 bytes in 459 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:04:25.949261 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b61a286-bb99-449f-99bc-abaf50798e8c None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:25.949621 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b61a286-bb99-449f-99bc-abaf50798e8c None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:25.949968 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b61a286-bb99-449f-99bc-abaf50798e8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:25.970059 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-0b61a286-bb99-449f-99bc-abaf50798e8c None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:04:25.971265 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-0b61a286-bb99-449f-99bc-abaf50798e8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:25.979710 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1347/2674] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 10:04:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:04:25.987395 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0064584-fa98-41af-af1a-f6203dde7518 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:25.987713 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0064584-fa98-41af-af1a-f6203dde7518 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:25.987994 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0064584-fa98-41af-af1a-f6203dde7518 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:26.009709 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-a0064584-fa98-41af-af1a-f6203dde7518 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:04:26.010901 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-a0064584-fa98-41af-af1a-f6203dde7518 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:26.019048 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1348/2675] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 10:04:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:04:26.023229 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0064584-fa98-41af-af1a-f6203dde7518 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:26.032317 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0064584-fa98-41af-af1a-f6203dde7518 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:26.037822 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b90a629-243a-41ce-ab05-d65508283498 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:26.038398 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b90a629-243a-41ce-ab05-d65508283498 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:26.038672 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b90a629-243a-41ce-ab05-d65508283498 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:26.038942 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b90a629-243a-41ce-ab05-d65508283498 admin admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:26.039576 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b90a629-243a-41ce-ab05-d65508283498 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:04:26.039905 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b90a629-243a-41ce-ab05-d65508283498 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:26.041487 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b90a629-243a-41ce-ab05-d65508283498 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:26.053506 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1349/2676] 10.222.0.16 () {58 vars in 1153 bytes} [Wed Aug 24 10:04:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 10:04:26.066570 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b90a629-243a-41ce-ab05-d65508283498 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:26.076378 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b90a629-243a-41ce-ab05-d65508283498 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:26.081348 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce7d8bd-ff21-4b66-98b9-b9b6c41f845e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:26.081936 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce7d8bd-ff21-4b66-98b9-b9b6c41f845e admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:26.082219 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce7d8bd-ff21-4b66-98b9-b9b6c41f845e admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:26.082529 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce7d8bd-ff21-4b66-98b9-b9b6c41f845e admin admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:26.083180 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce7d8bd-ff21-4b66-98b9-b9b6c41f845e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:26.085634 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce7d8bd-ff21-4b66-98b9-b9b6c41f845e admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:26.114947 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-ea990299-5dd6-4f9d-8f6f-d1457bd2dbcd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aad8f8b8a6cf48019b7f987bd5d1cd3c'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:26.115727 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-ea990299-5dd6-4f9d-8f6f-d1457bd2dbcd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aad8f8b8a6cf48019b7f987bd5d1cd3c'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:26.125869 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:04:26.128061 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-ea990299-5dd6-4f9d-8f6f-d1457bd2dbcd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aad8f8b8a6cf48019b7f987bd5d1cd3c'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:26.134864 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-ea990299-5dd6-4f9d-8f6f-d1457bd2dbcd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aad8f8b8a6cf48019b7f987bd5d1cd3c'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:26.183565 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1350/2677] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 24 10:04:26 2022] POST /identity/v3/projects => generated 385 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 10:04:26.186847 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce7d8bd-ff21-4b66-98b9-b9b6c41f845e admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:26.233257 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:04:26.244288 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-ea990299-5dd6-4f9d-8f6f-d1457bd2dbcd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aad8f8b8a6cf48019b7f987bd5d1cd3c'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:26.251308 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1328/2678] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 24 10:04:25 2022] DELETE /identity/v3/users/aad8f8b8a6cf48019b7f987bd5d1cd3c => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 10:04:26.255782 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce7d8bd-ff21-4b66-98b9-b9b6c41f845e admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:26.320545 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d244243-eb84-414a-bb75-7f8b346f6dce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:26.321117 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d244243-eb84-414a-bb75-7f8b346f6dce admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:26.321379 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d244243-eb84-414a-bb75-7f8b346f6dce admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:26.321649 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d244243-eb84-414a-bb75-7f8b346f6dce admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:26.322298 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d244243-eb84-414a-bb75-7f8b346f6dce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:26.324260 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d244243-eb84-414a-bb75-7f8b346f6dce admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:26.546085 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1351/2679] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 24 10:04:26 2022] POST /identity/v3/users => generated 293 bytes in 357 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 10:04:26.547271 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea990299-5dd6-4f9d-8f6f-d1457bd2dbcd admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:26.548726 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-ea990299-5dd6-4f9d-8f6f-d1457bd2dbcd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:26.562660 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea990299-5dd6-4f9d-8f6f-d1457bd2dbcd admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:26.567286 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a7c68b-78bc-49e4-9bd4-b0dcfe509df2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:26.568420 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a7c68b-78bc-49e4-9bd4-b0dcfe509df2 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:26.568763 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a7c68b-78bc-49e4-9bd4-b0dcfe509df2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:26.569037 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a7c68b-78bc-49e4-9bd4-b0dcfe509df2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:26.569704 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a7c68b-78bc-49e4-9bd4-b0dcfe509df2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:26.571319 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a7c68b-78bc-49e4-9bd4-b0dcfe509df2 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:26.582849 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1329/2680] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 24 10:04:26 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 10:04:26.587282 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d244243-eb84-414a-bb75-7f8b346f6dce admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:26.611436 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d244243-eb84-414a-bb75-7f8b346f6dce admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:26.617378 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02164cc5-311a-4767-8553-9a4d7b6fc7c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:26.618089 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02164cc5-311a-4767-8553-9a4d7b6fc7c1 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:26.618375 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02164cc5-311a-4767-8553-9a4d7b6fc7c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:26.618644 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02164cc5-311a-4767-8553-9a4d7b6fc7c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:26.619379 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02164cc5-311a-4767-8553-9a4d7b6fc7c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:26.621114 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02164cc5-311a-4767-8553-9a4d7b6fc7c1 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:26.632596 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1352/2681] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 24 10:04:26 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:04:26.636354 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a7c68b-78bc-49e4-9bd4-b0dcfe509df2 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:26.646540 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a7c68b-78bc-49e4-9bd4-b0dcfe509df2 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:26.651370 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e761419-969f-490d-ad27-008755c63d95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:26.652158 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e761419-969f-490d-ad27-008755c63d95 admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:26.652513 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e761419-969f-490d-ad27-008755c63d95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:26.652856 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e761419-969f-490d-ad27-008755c63d95 admin admin] PATH_INFO: `/v3/projects/54e30c94b2b147cb92a2e5037ba71131/users/6ea6558de7a7474e8258cd7fa55617de/roles/136cdf78e00a426997dd4e37faff489b` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:26.669182 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e761419-969f-490d-ad27-008755c63d95 admin admin] RBAC: Authorizing `identity:create_grant(project_id=54e30c94b2b147cb92a2e5037ba71131, user_id=6ea6558de7a7474e8258cd7fa55617de, role_id=136cdf78e00a426997dd4e37faff489b)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:26.671958 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e761419-969f-490d-ad27-008755c63d95 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:26.788374 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1330/2682] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 24 10:04:26 2022] PUT /identity/v3/projects/54e30c94b2b147cb92a2e5037ba71131/users/6ea6558de7a7474e8258cd7fa55617de/roles/136cdf78e00a426997dd4e37faff489b => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:04:26.801862 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea28b03e-185b-46d1-b561-f8fd59456c29 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:26.802305 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea28b03e-185b-46d1-b561-f8fd59456c29 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:26.803184 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea28b03e-185b-46d1-b561-f8fd59456c29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:26.831605 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-ea28b03e-185b-46d1-b561-f8fd59456c29 None None] MFA Rules not processed for user `6ea6558de7a7474e8258cd7fa55617de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:04:26.832825 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-ea28b03e-185b-46d1-b561-f8fd59456c29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:26.874620 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e761419-969f-490d-ad27-008755c63d95 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:26.876631 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-3e761419-969f-490d-ad27-008755c63d95 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:26.906323 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1353/2683] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 10:04:26 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:04:26.928862 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea28b03e-185b-46d1-b561-f8fd59456c29 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:26.930654 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-ea28b03e-185b-46d1-b561-f8fd59456c29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:26.961397 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e761419-969f-490d-ad27-008755c63d95 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:26.966528 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbeb14a-accd-4411-88bc-85cda4730716 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:26.967208 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbeb14a-accd-4411-88bc-85cda4730716 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:26.967441 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbeb14a-accd-4411-88bc-85cda4730716 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:26.967746 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbeb14a-accd-4411-88bc-85cda4730716 admin admin] PATH_INFO: `/v3/projects/6802fafdf54247fa9d0a495c26809ba4` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:26.968882 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbeb14a-accd-4411-88bc-85cda4730716 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6802fafdf54247fa9d0a495c26809ba4)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:26.970767 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbeb14a-accd-4411-88bc-85cda4730716 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:26.989321 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea28b03e-185b-46d1-b561-f8fd59456c29 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:26.993157 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3c4c84-18dc-4e2b-9e81-3f8998789202 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c32b83577c46488906fe409a790b24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:26.993772 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3c4c84-18dc-4e2b-9e81-3f8998789202 service cinder] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:26.994057 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3c4c84-18dc-4e2b-9e81-3f8998789202 service cinder] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:26.994334 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3c4c84-18dc-4e2b-9e81-3f8998789202 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:27.003271 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3c4c84-18dc-4e2b-9e81-3f8998789202 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:27.004848 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3c4c84-18dc-4e2b-9e81-3f8998789202 service cinder] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:27.008443 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:04:27.014493 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1354/2684] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 10:04:26 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 10:04:27.069126 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-fcbeb14a-accd-4411-88bc-85cda4730716 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6802fafdf54247fa9d0a495c26809ba4', 'request_id': 'req-fcbeb14a-accd-4411-88bc-85cda4730716'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:27.077974 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-fcbeb14a-accd-4411-88bc-85cda4730716 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6802fafdf54247fa9d0a495c26809ba4', 'request_id': 'req-fcbeb14a-accd-4411-88bc-85cda4730716'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:27.086637 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-fcbeb14a-accd-4411-88bc-85cda4730716 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6802fafdf54247fa9d0a495c26809ba4', 'request_id': 'req-fcbeb14a-accd-4411-88bc-85cda4730716'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:27.198949 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-fcbeb14a-accd-4411-88bc-85cda4730716 admin admin] The token cache is being invalidate because project 6802fafdf54247fa9d0a495c26809ba4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:04:27.199249 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-fcbeb14a-accd-4411-88bc-85cda4730716 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:27.199939 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-fcbeb14a-accd-4411-88bc-85cda4730716 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:27.201503 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1331/2685] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 24 10:04:26 2022] DELETE /identity/v3/projects/6802fafdf54247fa9d0a495c26809ba4 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 10:04:27.207677 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa42abb-6cd0-405c-99da-623fcad49091 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:27.207967 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa42abb-6cd0-405c-99da-623fcad49091 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:27.208240 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa42abb-6cd0-405c-99da-623fcad49091 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:27.229574 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-7aa42abb-6cd0-405c-99da-623fcad49091 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:04:27.230908 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-7aa42abb-6cd0-405c-99da-623fcad49091 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:27.294680 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1355/2686] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 10:04:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:04:27.303169 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b699d2bc-eb43-4f15-8384-e9ef75c6c963 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:27.303405 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b699d2bc-eb43-4f15-8384-e9ef75c6c963 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:27.303679 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b699d2bc-eb43-4f15-8384-e9ef75c6c963 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:27.324421 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-b699d2bc-eb43-4f15-8384-e9ef75c6c963 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:04:27.325730 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-b699d2bc-eb43-4f15-8384-e9ef75c6c963 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:27.334765 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1332/2687] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 10:04:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:04:27.338900 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa42abb-6cd0-405c-99da-623fcad49091 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:27.349368 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa42abb-6cd0-405c-99da-623fcad49091 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:27.353780 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be99413-d9f8-46ee-bbfd-59d0b2418004 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:27.354386 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be99413-d9f8-46ee-bbfd-59d0b2418004 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:27.354670 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be99413-d9f8-46ee-bbfd-59d0b2418004 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:27.354947 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be99413-d9f8-46ee-bbfd-59d0b2418004 admin admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:27.355638 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be99413-d9f8-46ee-bbfd-59d0b2418004 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:04:27.355948 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be99413-d9f8-46ee-bbfd-59d0b2418004 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:27.357547 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be99413-d9f8-46ee-bbfd-59d0b2418004 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:27.369280 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1356/2688] 10.222.0.16 () {58 vars in 1153 bytes} [Wed Aug 24 10:04:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:04:27.373988 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b699d2bc-eb43-4f15-8384-e9ef75c6c963 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:27.384078 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b699d2bc-eb43-4f15-8384-e9ef75c6c963 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:27.389241 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadbd935-5f47-408b-9ea7-2bb72aa2f777 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:27.389855 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadbd935-5f47-408b-9ea7-2bb72aa2f777 admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:27.390790 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadbd935-5f47-408b-9ea7-2bb72aa2f777 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:27.390790 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadbd935-5f47-408b-9ea7-2bb72aa2f777 admin admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:27.391156 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadbd935-5f47-408b-9ea7-2bb72aa2f777 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:27.393034 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadbd935-5f47-408b-9ea7-2bb72aa2f777 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:27.491778 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1333/2689] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 24 10:04:27 2022] POST /identity/v3/projects => generated 381 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:04:27.495679 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be99413-d9f8-46ee-bbfd-59d0b2418004 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:27.506207 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be99413-d9f8-46ee-bbfd-59d0b2418004 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:27.568459 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa663b3-e523-4185-a255-1c594d0bfee5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:27.569171 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa663b3-e523-4185-a255-1c594d0bfee5 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:27.569282 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa663b3-e523-4185-a255-1c594d0bfee5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:27.569462 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa663b3-e523-4185-a255-1c594d0bfee5 admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:27.570097 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa663b3-e523-4185-a255-1c594d0bfee5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:27.572100 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa663b3-e523-4185-a255-1c594d0bfee5 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:27.716242 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1357/2690] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 24 10:04:27 2022] POST /identity/v3/users => generated 291 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:04:27.717431 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadbd935-5f47-408b-9ea7-2bb72aa2f777 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:27.732135 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:DELETE=9,keystone:SELECT=87,keystone:INSERT=4 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:04:27.732135 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadbd935-5f47-408b-9ea7-2bb72aa2f777 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:27.734569 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208ede80-5005-4d7e-8468-5d2c924c0e48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:27.734851 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208ede80-5005-4d7e-8468-5d2c924c0e48 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:27.735093 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208ede80-5005-4d7e-8468-5d2c924c0e48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:27.735368 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208ede80-5005-4d7e-8468-5d2c924c0e48 admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:27.736073 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208ede80-5005-4d7e-8468-5d2c924c0e48 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:27.737706 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208ede80-5005-4d7e-8468-5d2c924c0e48 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:27.749125 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1334/2691] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 24 10:04:27 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:04:27.753301 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa663b3-e523-4185-a255-1c594d0bfee5 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:27.765090 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208ede80-5005-4d7e-8468-5d2c924c0e48 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:27.766387 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa663b3-e523-4185-a255-1c594d0bfee5 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:27.766772 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-208ede80-5005-4d7e-8468-5d2c924c0e48 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:27.771215 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfdb03e-a2dd-4870-9e02-7a2660a7777b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:27.771857 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfdb03e-a2dd-4870-9e02-7a2660a7777b admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:27.772298 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfdb03e-a2dd-4870-9e02-7a2660a7777b admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:27.772646 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfdb03e-a2dd-4870-9e02-7a2660a7777b admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:27.773465 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfdb03e-a2dd-4870-9e02-7a2660a7777b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:27.775408 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfdb03e-a2dd-4870-9e02-7a2660a7777b admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:27.780261 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208ede80-5005-4d7e-8468-5d2c924c0e48 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:27.796866 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1358/2692] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 24 10:04:27 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:04:27.796976 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5ce411-1818-4130-a351-e533351fadf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:27.796976 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5ce411-1818-4130-a351-e533351fadf7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:27.796976 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5ce411-1818-4130-a351-e533351fadf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:27.796976 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5ce411-1818-4130-a351-e533351fadf7 admin admin] PATH_INFO: `/v3/users/5d45e9fcb8244b4ab91dfd251ae84216` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:27.797345 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfdb03e-a2dd-4870-9e02-7a2660a7777b admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:27.797444 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5ce411-1818-4130-a351-e533351fadf7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5d45e9fcb8244b4ab91dfd251ae84216)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:27.797444 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5ce411-1818-4130-a351-e533351fadf7 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:27.801971 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfdb03e-a2dd-4870-9e02-7a2660a7777b admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:27.807274 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91b65fd-619a-4603-8afc-2a1049454586 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:27.808125 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91b65fd-619a-4603-8afc-2a1049454586 admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:27.808409 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91b65fd-619a-4603-8afc-2a1049454586 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:27.808906 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91b65fd-619a-4603-8afc-2a1049454586 admin admin] PATH_INFO: `/v3/projects/d8a1ebc1209246e6a4e472cf816dff88/users/a2c670e02fa64a1a9390cc1908c5545a/roles/136cdf78e00a426997dd4e37faff489b` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:27.826691 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91b65fd-619a-4603-8afc-2a1049454586 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8a1ebc1209246e6a4e472cf816dff88, user_id=a2c670e02fa64a1a9390cc1908c5545a, role_id=136cdf78e00a426997dd4e37faff489b)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:27.827994 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91b65fd-619a-4603-8afc-2a1049454586 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:27.954059 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1359/2693] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 24 10:04:27 2022] PUT /identity/v3/projects/d8a1ebc1209246e6a4e472cf816dff88/users/a2c670e02fa64a1a9390cc1908c5545a/roles/136cdf78e00a426997dd4e37faff489b => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 10:04:27.966367 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a84b6d-86d0-42f8-8fba-58331523a9e8 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:27.966654 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a84b6d-86d0-42f8-8fba-58331523a9e8 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:27.966926 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a84b6d-86d0-42f8-8fba-58331523a9e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:27.996370 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-12a84b6d-86d0-42f8-8fba-58331523a9e8 None None] MFA Rules not processed for user `a2c670e02fa64a1a9390cc1908c5545a`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:04:27.997842 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-12a84b6d-86d0-42f8-8fba-58331523a9e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:28.029458 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:04:28.033272 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-be5ce411-1818-4130-a351-e533351fadf7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5d45e9fcb8244b4ab91dfd251ae84216'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:28.034730 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-be5ce411-1818-4130-a351-e533351fadf7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5d45e9fcb8244b4ab91dfd251ae84216'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:28.047723 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-be5ce411-1818-4130-a351-e533351fadf7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5d45e9fcb8244b4ab91dfd251ae84216'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:28.055205 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-be5ce411-1818-4130-a351-e533351fadf7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5d45e9fcb8244b4ab91dfd251ae84216'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:28.071026 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1360/2694] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 10:04:27 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:04:28.075656 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a84b6d-86d0-42f8-8fba-58331523a9e8 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:28.145654 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a84b6d-86d0-42f8-8fba-58331523a9e8 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:28.150933 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40abadb1-9261-4478-aaa4-4b7b1c7b2f0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:28.151614 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40abadb1-9261-4478-aaa4-4b7b1c7b2f0f admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:28.151905 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40abadb1-9261-4478-aaa4-4b7b1c7b2f0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:28.152177 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40abadb1-9261-4478-aaa4-4b7b1c7b2f0f admin admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:28.152803 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40abadb1-9261-4478-aaa4-4b7b1c7b2f0f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:28.154719 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40abadb1-9261-4478-aaa4-4b7b1c7b2f0f admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:28.164372 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-be5ce411-1818-4130-a351-e533351fadf7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5d45e9fcb8244b4ab91dfd251ae84216'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:28.171368 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1335/2695] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 24 10:04:27 2022] DELETE /identity/v3/users/5d45e9fcb8244b4ab91dfd251ae84216 => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 10:04:28.175855 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7fa545-d3f2-4b57-8596-32584177d0dd None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:28.176250 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7fa545-d3f2-4b57-8596-32584177d0dd None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:28.176597 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7fa545-d3f2-4b57-8596-32584177d0dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:28.197976 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-3b7fa545-d3f2-4b57-8596-32584177d0dd None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:04:28.199637 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-3b7fa545-d3f2-4b57-8596-32584177d0dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:28.264081 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1336/2696] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 10:04:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:04:28.271853 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1361/2697] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 24 10:04:28 2022] POST /identity/v3/projects => generated 383 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 10:04:28.274706 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40abadb1-9261-4478-aaa4-4b7b1c7b2f0f admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:28.274826 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7fa545-d3f2-4b57-8596-32584177d0dd None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:28.276319 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-3b7fa545-d3f2-4b57-8596-32584177d0dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:28.276428 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-40abadb1-9261-4478-aaa4-4b7b1c7b2f0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:28.352980 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7fa545-d3f2-4b57-8596-32584177d0dd None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:28.358651 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40abadb1-9261-4478-aaa4-4b7b1c7b2f0f admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:28.363908 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c6393a-2d28-400e-8a5f-4a4253114f0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:28.364548 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c6393a-2d28-400e-8a5f-4a4253114f0e admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:28.365426 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c6393a-2d28-400e-8a5f-4a4253114f0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:28.365703 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c6393a-2d28-400e-8a5f-4a4253114f0e admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:28.366321 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c6393a-2d28-400e-8a5f-4a4253114f0e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:28.368525 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c6393a-2d28-400e-8a5f-4a4253114f0e admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:28.374926 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:04:28.406727 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31b0b3d-a2b7-45ad-9e8a-459298ca6abe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e5260e798404e8bb12b25ca8ea97052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:28.407360 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31b0b3d-a2b7-45ad-9e8a-459298ca6abe service neutron] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:28.407666 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31b0b3d-a2b7-45ad-9e8a-459298ca6abe service neutron] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:28.408101 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31b0b3d-a2b7-45ad-9e8a-459298ca6abe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:28.417255 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31b0b3d-a2b7-45ad-9e8a-459298ca6abe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:28.418685 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31b0b3d-a2b7-45ad-9e8a-459298ca6abe service neutron] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:28.428894 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1337/2698] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 10:04:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:04:28.537579 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1362/2699] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 24 10:04:28 2022] POST /identity/v3/users => generated 298 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:04:28.541628 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31b0b3d-a2b7-45ad-9e8a-459298ca6abe service neutron] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:28.551783 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31b0b3d-a2b7-45ad-9e8a-459298ca6abe service neutron] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:28.557196 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6ed96c-3ee4-4a51-9e4b-4e89c7611258 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:28.558043 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6ed96c-3ee4-4a51-9e4b-4e89c7611258 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:28.558658 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6ed96c-3ee4-4a51-9e4b-4e89c7611258 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:28.559155 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6ed96c-3ee4-4a51-9e4b-4e89c7611258 admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:28.560365 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6ed96c-3ee4-4a51-9e4b-4e89c7611258 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:28.562242 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6ed96c-3ee4-4a51-9e4b-4e89c7611258 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:28.573433 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1338/2700] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 24 10:04:28 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:04:28.577669 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c6393a-2d28-400e-8a5f-4a4253114f0e admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:28.588798 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c6393a-2d28-400e-8a5f-4a4253114f0e admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:28.593765 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1773f9-e023-45e4-99e6-747082ec6c5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:28.594432 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1773f9-e023-45e4-99e6-747082ec6c5e admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:28.594707 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1773f9-e023-45e4-99e6-747082ec6c5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:28.594976 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1773f9-e023-45e4-99e6-747082ec6c5e admin admin] PATH_INFO: `/v3/projects/7e3da9bbdf1144c48b4a43fff59200b8/users/3e641bbb85744261abd1440a427c936c/roles/15f634d4a2d247ddab90c458e5e1c141` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:28.611263 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1773f9-e023-45e4-99e6-747082ec6c5e admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e3da9bbdf1144c48b4a43fff59200b8, user_id=3e641bbb85744261abd1440a427c936c, role_id=15f634d4a2d247ddab90c458e5e1c141)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:28.613856 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1773f9-e023-45e4-99e6-747082ec6c5e admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:28.750595 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1363/2701] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 24 10:04:28 2022] PUT /identity/v3/projects/7e3da9bbdf1144c48b4a43fff59200b8/users/3e641bbb85744261abd1440a427c936c/roles/15f634d4a2d247ddab90c458e5e1c141 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:04:28.752686 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6ed96c-3ee4-4a51-9e4b-4e89c7611258 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:28.763041 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6ed96c-3ee4-4a51-9e4b-4e89c7611258 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:28.827407 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3133736a-d3b0-4162-8b16-9a54fa8e63c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:28.828190 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3133736a-d3b0-4162-8b16-9a54fa8e63c5 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:28.828478 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3133736a-d3b0-4162-8b16-9a54fa8e63c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:28.828754 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3133736a-d3b0-4162-8b16-9a54fa8e63c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:28.829404 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3133736a-d3b0-4162-8b16-9a54fa8e63c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:28.831269 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3133736a-d3b0-4162-8b16-9a54fa8e63c5 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:28.843462 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1339/2702] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 24 10:04:28 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:04:28.849873 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1773f9-e023-45e4-99e6-747082ec6c5e admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:28.869187 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1773f9-e023-45e4-99e6-747082ec6c5e admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:28.874194 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49196822-920e-43b7-bc05-55788ca958b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:28.874859 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49196822-920e-43b7-bc05-55788ca958b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:28.875235 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49196822-920e-43b7-bc05-55788ca958b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:28.875532 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49196822-920e-43b7-bc05-55788ca958b7 admin admin] PATH_INFO: `/v3/projects/7e3da9bbdf1144c48b4a43fff59200b8/users/3e641bbb85744261abd1440a427c936c/roles/15f634d4a2d247ddab90c458e5e1c141` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:28.877557 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49196822-920e-43b7-bc05-55788ca958b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e3da9bbdf1144c48b4a43fff59200b8, user_id=3e641bbb85744261abd1440a427c936c, role_id=15f634d4a2d247ddab90c458e5e1c141)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:28.880415 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49196822-920e-43b7-bc05-55788ca958b7 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:28.890907 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1364/2703] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 24 10:04:28 2022] PUT /identity/v3/projects/7e3da9bbdf1144c48b4a43fff59200b8/users/3e641bbb85744261abd1440a427c936c/roles/15f634d4a2d247ddab90c458e5e1c141 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:04:28.901199 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b03bf5-4ef9-4df3-a558-1010e365134e None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:28.901493 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b03bf5-4ef9-4df3-a558-1010e365134e None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:28.901772 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b03bf5-4ef9-4df3-a558-1010e365134e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:28.932137 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-43b03bf5-4ef9-4df3-a558-1010e365134e None None] MFA Rules not processed for user `3e641bbb85744261abd1440a427c936c`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:04:28.933371 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-43b03bf5-4ef9-4df3-a558-1010e365134e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:28.954769 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49196822-920e-43b7-bc05-55788ca958b7 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:28.957256 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-49196822-920e-43b7-bc05-55788ca958b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:28.969079 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:04:29.023835 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1340/2704] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 10:04:28 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:04:29.036337 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b03bf5-4ef9-4df3-a558-1010e365134e None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:29.038132 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-43b03bf5-4ef9-4df3-a558-1010e365134e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:29.040554 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49196822-920e-43b7-bc05-55788ca958b7 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:29.046048 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9305ed-6978-4b90-b146-4aad30ff8e34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:29.046688 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9305ed-6978-4b90-b146-4aad30ff8e34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:29.046967 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9305ed-6978-4b90-b146-4aad30ff8e34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:29.047236 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9305ed-6978-4b90-b146-4aad30ff8e34 admin admin] PATH_INFO: `/v3/projects/160be357dd814b05951273708265833f` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:29.048492 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9305ed-6978-4b90-b146-4aad30ff8e34 admin admin] RBAC: Authorizing `identity:delete_project(project_id=160be357dd814b05951273708265833f)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:29.050394 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9305ed-6978-4b90-b146-4aad30ff8e34 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:29.097973 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b03bf5-4ef9-4df3-a558-1010e365134e None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:29.102276 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4974f5dc-4095-464a-9a76-6ac0b18e4ef8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c32b83577c46488906fe409a790b24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:29.102899 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4974f5dc-4095-464a-9a76-6ac0b18e4ef8 service cinder] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:29.103178 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4974f5dc-4095-464a-9a76-6ac0b18e4ef8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:29.103448 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4974f5dc-4095-464a-9a76-6ac0b18e4ef8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:29.105285 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-4974f5dc-4095-464a-9a76-6ac0b18e4ef8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:29.164144 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4974f5dc-4095-464a-9a76-6ac0b18e4ef8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:29.165525 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4974f5dc-4095-464a-9a76-6ac0b18e4ef8 service cinder] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:29.174428 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1341/2705] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 10:04:29 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:04:29.247542 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-ab9305ed-6978-4b90-b146-4aad30ff8e34 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '160be357dd814b05951273708265833f', 'request_id': 'req-ab9305ed-6978-4b90-b146-4aad30ff8e34'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:29.254466 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-ab9305ed-6978-4b90-b146-4aad30ff8e34 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '160be357dd814b05951273708265833f', 'request_id': 'req-ab9305ed-6978-4b90-b146-4aad30ff8e34'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:29.261287 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-ab9305ed-6978-4b90-b146-4aad30ff8e34 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '160be357dd814b05951273708265833f', 'request_id': 'req-ab9305ed-6978-4b90-b146-4aad30ff8e34'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:29.381902 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4974f5dc-4095-464a-9a76-6ac0b18e4ef8 service cinder] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:29.383455 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-4974f5dc-4095-464a-9a76-6ac0b18e4ef8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:29.433940 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-ab9305ed-6978-4b90-b146-4aad30ff8e34 admin admin] The token cache is being invalidate because project 160be357dd814b05951273708265833f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81602) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:04:29.433940 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-ab9305ed-6978-4b90-b146-4aad30ff8e34 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:29.433940 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-ab9305ed-6978-4b90-b146-4aad30ff8e34 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:29.436147 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1365/2706] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 24 10:04:28 2022] DELETE /identity/v3/projects/160be357dd814b05951273708265833f => generated 0 bytes in 482 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:04:29.441973 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc96206-075d-4738-bd6a-f79bf1128b45 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:29.443652 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc96206-075d-4738-bd6a-f79bf1128b45 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:29.444179 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc96206-075d-4738-bd6a-f79bf1128b45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:29.454589 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4974f5dc-4095-464a-9a76-6ac0b18e4ef8 service cinder] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:29.481394 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-5cc96206-075d-4738-bd6a-f79bf1128b45 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:04:29.483443 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-5cc96206-075d-4738-bd6a-f79bf1128b45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:29.521637 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb001d8-d1bb-4068-95da-b7bed440f0ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e5260e798404e8bb12b25ca8ea97052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:29.522393 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb001d8-d1bb-4068-95da-b7bed440f0ad service neutron] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:29.522688 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb001d8-d1bb-4068-95da-b7bed440f0ad service neutron] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:29.523117 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb001d8-d1bb-4068-95da-b7bed440f0ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:29.525099 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-6bb001d8-d1bb-4068-95da-b7bed440f0ad service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:29.536175 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb001d8-d1bb-4068-95da-b7bed440f0ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:29.538967 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb001d8-d1bb-4068-95da-b7bed440f0ad service neutron] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:29.550791 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1342/2707] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 10:04:29 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 10:04:29.563216 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:04:29.576974 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1366/2708] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 10:04:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:04:29.585430 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53460863-4ccd-4eea-ae4f-c24381c4f709 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:29.585705 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53460863-4ccd-4eea-ae4f-c24381c4f709 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:29.586010 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53460863-4ccd-4eea-ae4f-c24381c4f709 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:29.612596 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-53460863-4ccd-4eea-ae4f-c24381c4f709 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:04:29.613912 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-53460863-4ccd-4eea-ae4f-c24381c4f709 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:29.623433 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1343/2709] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 10:04:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:04:29.627790 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc96206-075d-4738-bd6a-f79bf1128b45 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:29.638130 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc96206-075d-4738-bd6a-f79bf1128b45 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:29.643128 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2caf8a-f591-43e2-aa22-d5d628241fac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:29.644179 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2caf8a-f591-43e2-aa22-d5d628241fac admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:29.644179 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2caf8a-f591-43e2-aa22-d5d628241fac admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:29.644377 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2caf8a-f591-43e2-aa22-d5d628241fac admin admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:29.644989 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2caf8a-f591-43e2-aa22-d5d628241fac admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:04:29.645287 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2caf8a-f591-43e2-aa22-d5d628241fac admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:29.647360 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2caf8a-f591-43e2-aa22-d5d628241fac admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:29.659639 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1367/2710] 10.222.0.16 () {58 vars in 1153 bytes} [Wed Aug 24 10:04:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:04:29.668887 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53460863-4ccd-4eea-ae4f-c24381c4f709 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:29.681037 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53460863-4ccd-4eea-ae4f-c24381c4f709 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:29.688933 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f01d7fe-1a35-4ba7-a7f7-f5c1b1ece2d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:29.689569 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f01d7fe-1a35-4ba7-a7f7-f5c1b1ece2d2 admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:29.690601 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f01d7fe-1a35-4ba7-a7f7-f5c1b1ece2d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:29.690957 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f01d7fe-1a35-4ba7-a7f7-f5c1b1ece2d2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:29.691745 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f01d7fe-1a35-4ba7-a7f7-f5c1b1ece2d2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:29.694761 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f01d7fe-1a35-4ba7-a7f7-f5c1b1ece2d2 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:29.783615 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1344/2711] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 24 10:04:29 2022] POST /identity/v3/projects => generated 383 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:04:29.788536 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2caf8a-f591-43e2-aa22-d5d628241fac admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:29.799451 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2caf8a-f591-43e2-aa22-d5d628241fac admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:29.863122 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:04:29.911498 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04df651c-f6e5-4e2b-9347-57f4d6893046 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:29.921207 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04df651c-f6e5-4e2b-9347-57f4d6893046 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:29.921207 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04df651c-f6e5-4e2b-9347-57f4d6893046 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:29.921506 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04df651c-f6e5-4e2b-9347-57f4d6893046 admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:29.922150 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04df651c-f6e5-4e2b-9347-57f4d6893046 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:29.928515 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04df651c-f6e5-4e2b-9347-57f4d6893046 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:30.027766 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1368/2712] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 24 10:04:29 2022] POST /identity/v3/users => generated 292 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:04:30.029214 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f01d7fe-1a35-4ba7-a7f7-f5c1b1ece2d2 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:30.041945 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f01d7fe-1a35-4ba7-a7f7-f5c1b1ece2d2 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:30.051003 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcc3187-53ec-4f1b-8089-50cab1c80f1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:30.051718 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcc3187-53ec-4f1b-8089-50cab1c80f1f admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:30.052126 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcc3187-53ec-4f1b-8089-50cab1c80f1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:30.052970 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcc3187-53ec-4f1b-8089-50cab1c80f1f admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:30.054031 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcc3187-53ec-4f1b-8089-50cab1c80f1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:30.055793 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcc3187-53ec-4f1b-8089-50cab1c80f1f admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:30.078161 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1345/2713] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 24 10:04:30 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:04:30.078779 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04df651c-f6e5-4e2b-9347-57f4d6893046 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:30.091856 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04df651c-f6e5-4e2b-9347-57f4d6893046 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:30.097733 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f249e60f-6e4f-4700-a01d-688bd39c7680 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:30.099128 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f249e60f-6e4f-4700-a01d-688bd39c7680 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:30.100090 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f249e60f-6e4f-4700-a01d-688bd39c7680 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:30.100635 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f249e60f-6e4f-4700-a01d-688bd39c7680 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:30.101405 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f249e60f-6e4f-4700-a01d-688bd39c7680 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:30.103263 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f249e60f-6e4f-4700-a01d-688bd39c7680 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:30.118549 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1369/2714] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 24 10:04:30 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:04:30.123408 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcc3187-53ec-4f1b-8089-50cab1c80f1f admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:30.135757 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcc3187-53ec-4f1b-8089-50cab1c80f1f admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:30.142829 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03945d9-a735-44cc-bc97-5690f22f8880 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:30.143526 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03945d9-a735-44cc-bc97-5690f22f8880 admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:30.143856 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03945d9-a735-44cc-bc97-5690f22f8880 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:30.144201 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03945d9-a735-44cc-bc97-5690f22f8880 admin admin] PATH_INFO: `/v3/projects/428f4e988b10492c87dddc5a0c4f34fb/users/ea68de414dcd4ff79bfd0a1305c58f63/roles/136cdf78e00a426997dd4e37faff489b` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:30.148036 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:04:30.162799 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03945d9-a735-44cc-bc97-5690f22f8880 admin admin] RBAC: Authorizing `identity:create_grant(project_id=428f4e988b10492c87dddc5a0c4f34fb, user_id=ea68de414dcd4ff79bfd0a1305c58f63, role_id=136cdf78e00a426997dd4e37faff489b)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:30.168310 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03945d9-a735-44cc-bc97-5690f22f8880 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:30.280308 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1346/2715] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 24 10:04:30 2022] PUT /identity/v3/projects/428f4e988b10492c87dddc5a0c4f34fb/users/ea68de414dcd4ff79bfd0a1305c58f63/roles/136cdf78e00a426997dd4e37faff489b => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:04:30.287797 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e38fc5-2cf1-4737-8342-59e34b4b568e None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:30.288089 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e38fc5-2cf1-4737-8342-59e34b4b568e None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:30.288418 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e38fc5-2cf1-4737-8342-59e34b4b568e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:30.313000 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-f1e38fc5-2cf1-4737-8342-59e34b4b568e None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:04:30.314233 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-f1e38fc5-2cf1-4737-8342-59e34b4b568e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:30.401660 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1370/2716] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 10:04:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:04:30.411959 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03945d9-a735-44cc-bc97-5690f22f8880 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:30.413825 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-b03945d9-a735-44cc-bc97-5690f22f8880 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:30.480924 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03945d9-a735-44cc-bc97-5690f22f8880 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:30.484687 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bf2e36-718d-4502-ad96-a133d0f42348 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e5260e798404e8bb12b25ca8ea97052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:30.485305 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bf2e36-718d-4502-ad96-a133d0f42348 service neutron] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:30.485604 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bf2e36-718d-4502-ad96-a133d0f42348 service neutron] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:30.485876 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bf2e36-718d-4502-ad96-a133d0f42348 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:30.495394 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0bf2e36-718d-4502-ad96-a133d0f42348 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:30.496962 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0bf2e36-718d-4502-ad96-a133d0f42348 service neutron] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:30.506896 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1347/2717] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 10:04:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:04:31.513570 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e38fc5-2cf1-4737-8342-59e34b4b568e None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:31.515030 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-f1e38fc5-2cf1-4737-8342-59e34b4b568e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:31.527163 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e38fc5-2cf1-4737-8342-59e34b4b568e None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:31.531812 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf95d7a-d649-4cb2-81d4-2ab95f56f80f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:31.532441 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf95d7a-d649-4cb2-81d4-2ab95f56f80f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:31.532728 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf95d7a-d649-4cb2-81d4-2ab95f56f80f admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:31.533004 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf95d7a-d649-4cb2-81d4-2ab95f56f80f admin admin] PATH_INFO: `/v3/users/6ea6558de7a7474e8258cd7fa55617de` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:31.534220 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf95d7a-d649-4cb2-81d4-2ab95f56f80f admin admin] RBAC: Authorizing `identity:delete_user(user_id=6ea6558de7a7474e8258cd7fa55617de)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:31.536291 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf95d7a-d649-4cb2-81d4-2ab95f56f80f admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:31.806847 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-baf95d7a-d649-4cb2-81d4-2ab95f56f80f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6ea6558de7a7474e8258cd7fa55617de'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:31.807499 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-baf95d7a-d649-4cb2-81d4-2ab95f56f80f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6ea6558de7a7474e8258cd7fa55617de'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:31.894243 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-baf95d7a-d649-4cb2-81d4-2ab95f56f80f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6ea6558de7a7474e8258cd7fa55617de'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:31.894896 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-baf95d7a-d649-4cb2-81d4-2ab95f56f80f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6ea6558de7a7474e8258cd7fa55617de'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:31.902695 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-baf95d7a-d649-4cb2-81d4-2ab95f56f80f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6ea6558de7a7474e8258cd7fa55617de'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:31.920772 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1371/2718] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 24 10:04:31 2022] DELETE /identity/v3/users/6ea6558de7a7474e8258cd7fa55617de => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:04:31.925287 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa79cad-07f2-49f0-869e-51808513680f None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:31.925751 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa79cad-07f2-49f0-869e-51808513680f None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:31.925842 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa79cad-07f2-49f0-869e-51808513680f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:31.947515 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-5aa79cad-07f2-49f0-869e-51808513680f None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:04:31.948818 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-5aa79cad-07f2-49f0-869e-51808513680f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:32.013089 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1348/2719] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 10:04:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:04:32.023085 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf95d7a-d649-4cb2-81d4-2ab95f56f80f admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:32.024741 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-baf95d7a-d649-4cb2-81d4-2ab95f56f80f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:32.045165 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:04:32.212942 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf95d7a-d649-4cb2-81d4-2ab95f56f80f admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:32.216735 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcaa19b-8a62-4789-a29a-674ef071b7c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e5260e798404e8bb12b25ca8ea97052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:32.217348 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcaa19b-8a62-4789-a29a-674ef071b7c9 service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:32.217751 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcaa19b-8a62-4789-a29a-674ef071b7c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:32.218033 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcaa19b-8a62-4789-a29a-674ef071b7c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:32.227995 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcaa19b-8a62-4789-a29a-674ef071b7c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:32.229429 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcaa19b-8a62-4789-a29a-674ef071b7c9 service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:32.238986 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1372/2720] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 10:04:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:04:32.969052 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa79cad-07f2-49f0-869e-51808513680f None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:32.969052 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-5aa79cad-07f2-49f0-869e-51808513680f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:32.983082 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa79cad-07f2-49f0-869e-51808513680f None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:32.991812 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4876c50a-c1ce-41fa-b074-96deeed872b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:32.992776 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4876c50a-c1ce-41fa-b074-96deeed872b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:32.993230 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4876c50a-c1ce-41fa-b074-96deeed872b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:32.993669 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4876c50a-c1ce-41fa-b074-96deeed872b2 admin admin] PATH_INFO: `/v3/projects/54e30c94b2b147cb92a2e5037ba71131` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:32.995606 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4876c50a-c1ce-41fa-b074-96deeed872b2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=54e30c94b2b147cb92a2e5037ba71131)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:32.998967 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4876c50a-c1ce-41fa-b074-96deeed872b2 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:33.155916 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-4876c50a-c1ce-41fa-b074-96deeed872b2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '54e30c94b2b147cb92a2e5037ba71131', 'request_id': 'req-4876c50a-c1ce-41fa-b074-96deeed872b2'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:33.163668 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-4876c50a-c1ce-41fa-b074-96deeed872b2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '54e30c94b2b147cb92a2e5037ba71131', 'request_id': 'req-4876c50a-c1ce-41fa-b074-96deeed872b2'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:33.170495 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-4876c50a-c1ce-41fa-b074-96deeed872b2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '54e30c94b2b147cb92a2e5037ba71131', 'request_id': 'req-4876c50a-c1ce-41fa-b074-96deeed872b2'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:33.314737 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-4876c50a-c1ce-41fa-b074-96deeed872b2 admin admin] The token cache is being invalidate because project 54e30c94b2b147cb92a2e5037ba71131 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:04:33.315043 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-4876c50a-c1ce-41fa-b074-96deeed872b2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:33.315750 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-4876c50a-c1ce-41fa-b074-96deeed872b2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:33.317301 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1349/2721] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 24 10:04:32 2022] DELETE /identity/v3/projects/54e30c94b2b147cb92a2e5037ba71131 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:04:36.949737 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f858b3-845f-462e-bc1b-966223aa7875 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:36.950225 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f858b3-845f-462e-bc1b-966223aa7875 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:36.950225 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f858b3-845f-462e-bc1b-966223aa7875 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:36.986323 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-b7f858b3-845f-462e-bc1b-966223aa7875 None None] MFA Rules not processed for user `ea68de414dcd4ff79bfd0a1305c58f63`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:04:36.987964 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-b7f858b3-845f-462e-bc1b-966223aa7875 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:37.064185 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1373/2722] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 10:04:36 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:04:37.068899 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4876c50a-c1ce-41fa-b074-96deeed872b2 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:37.070433 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-4876c50a-c1ce-41fa-b074-96deeed872b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:37.085816 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:04:37.138920 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4876c50a-c1ce-41fa-b074-96deeed872b2 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:37.146293 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e058c5b-3a00-4a0b-9914-7ccce6ee18a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:37.146924 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e058c5b-3a00-4a0b-9914-7ccce6ee18a2 admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:37.151071 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e058c5b-3a00-4a0b-9914-7ccce6ee18a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:37.151071 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e058c5b-3a00-4a0b-9914-7ccce6ee18a2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:37.151071 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e058c5b-3a00-4a0b-9914-7ccce6ee18a2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:37.151071 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e058c5b-3a00-4a0b-9914-7ccce6ee18a2 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:37.257755 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1350/2723] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 24 10:04:37 2022] POST /identity/v3/projects => generated 383 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:04:37.261480 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f858b3-845f-462e-bc1b-966223aa7875 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:37.272862 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f858b3-845f-462e-bc1b-966223aa7875 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:37.328857 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82a398c-be06-48cf-a2e2-3a15df0e2b57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:37.329473 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82a398c-be06-48cf-a2e2-3a15df0e2b57 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:37.329753 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82a398c-be06-48cf-a2e2-3a15df0e2b57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:37.330153 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82a398c-be06-48cf-a2e2-3a15df0e2b57 admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:37.331172 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82a398c-be06-48cf-a2e2-3a15df0e2b57 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:37.333110 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82a398c-be06-48cf-a2e2-3a15df0e2b57 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:37.473700 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1374/2724] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 24 10:04:37 2022] POST /identity/v3/users => generated 298 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:04:37.477864 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e058c5b-3a00-4a0b-9914-7ccce6ee18a2 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:37.488092 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e058c5b-3a00-4a0b-9914-7ccce6ee18a2 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:37.493989 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfad4d31-c490-4beb-827b-b28ae4ae96df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:37.494741 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfad4d31-c490-4beb-827b-b28ae4ae96df admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:37.495023 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfad4d31-c490-4beb-827b-b28ae4ae96df admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:37.495299 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfad4d31-c490-4beb-827b-b28ae4ae96df admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:37.515826 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfad4d31-c490-4beb-827b-b28ae4ae96df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:37.515826 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfad4d31-c490-4beb-827b-b28ae4ae96df admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:37.515826 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1351/2725] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 24 10:04:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 10:04:37.518729 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82a398c-be06-48cf-a2e2-3a15df0e2b57 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:37.529064 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82a398c-be06-48cf-a2e2-3a15df0e2b57 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:37.533672 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ce35cc-28fe-4df7-9664-116e16e866dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:37.534332 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ce35cc-28fe-4df7-9664-116e16e866dd admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:37.534613 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ce35cc-28fe-4df7-9664-116e16e866dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:37.534892 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ce35cc-28fe-4df7-9664-116e16e866dd admin admin] PATH_INFO: `/v3/projects/ef18494b41144715bd8791e8a3cab9ea/users/2cb599e8e13f4932bc27ab2770b8963c/roles/15f634d4a2d247ddab90c458e5e1c141` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:37.549605 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ce35cc-28fe-4df7-9664-116e16e866dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef18494b41144715bd8791e8a3cab9ea, user_id=2cb599e8e13f4932bc27ab2770b8963c, role_id=15f634d4a2d247ddab90c458e5e1c141)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:37.552773 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ce35cc-28fe-4df7-9664-116e16e866dd admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:37.708255 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1375/2726] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 24 10:04:37 2022] PUT /identity/v3/projects/ef18494b41144715bd8791e8a3cab9ea/users/2cb599e8e13f4932bc27ab2770b8963c/roles/15f634d4a2d247ddab90c458e5e1c141 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:04:37.712519 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfad4d31-c490-4beb-827b-b28ae4ae96df admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:37.723809 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfad4d31-c490-4beb-827b-b28ae4ae96df admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:37.782223 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175c1e40-6ab5-4764-a0a8-a7ae42b59abc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:37.782838 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175c1e40-6ab5-4764-a0a8-a7ae42b59abc admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:37.783118 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175c1e40-6ab5-4764-a0a8-a7ae42b59abc admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:37.783396 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175c1e40-6ab5-4764-a0a8-a7ae42b59abc admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:37.784112 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175c1e40-6ab5-4764-a0a8-a7ae42b59abc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:37.785706 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175c1e40-6ab5-4764-a0a8-a7ae42b59abc admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:37.797162 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1352/2727] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 24 10:04:37 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:04:37.801393 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ce35cc-28fe-4df7-9664-116e16e866dd admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:37.812459 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ce35cc-28fe-4df7-9664-116e16e866dd admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:37.817256 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dae02d-2fd1-43ae-9cb1-56015c9cb389 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:37.818036 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dae02d-2fd1-43ae-9cb1-56015c9cb389 admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:37.818382 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dae02d-2fd1-43ae-9cb1-56015c9cb389 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:37.818729 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dae02d-2fd1-43ae-9cb1-56015c9cb389 admin admin] PATH_INFO: `/v3/projects/ef18494b41144715bd8791e8a3cab9ea/users/2cb599e8e13f4932bc27ab2770b8963c/roles/15f634d4a2d247ddab90c458e5e1c141` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:37.820825 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61dae02d-2fd1-43ae-9cb1-56015c9cb389 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef18494b41144715bd8791e8a3cab9ea, user_id=2cb599e8e13f4932bc27ab2770b8963c, role_id=15f634d4a2d247ddab90c458e5e1c141)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:37.823801 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61dae02d-2fd1-43ae-9cb1-56015c9cb389 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:37.830405 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:04:37.835267 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1376/2728] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 24 10:04:37 2022] PUT /identity/v3/projects/ef18494b41144715bd8791e8a3cab9ea/users/2cb599e8e13f4932bc27ab2770b8963c/roles/15f634d4a2d247ddab90c458e5e1c141 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:04:38.340415 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175c1e40-6ab5-4764-a0a8-a7ae42b59abc admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:38.342220 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-175c1e40-6ab5-4764-a0a8-a7ae42b59abc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:38.403664 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175c1e40-6ab5-4764-a0a8-a7ae42b59abc admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:38.438289 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:04:38.453786 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14829824-afbe-4875-b4af-902ea9e1d5a3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c32b83577c46488906fe409a790b24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:38.454400 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14829824-afbe-4875-b4af-902ea9e1d5a3 service cinder] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:38.454687 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14829824-afbe-4875-b4af-902ea9e1d5a3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:38.454950 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14829824-afbe-4875-b4af-902ea9e1d5a3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:38.457030 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-14829824-afbe-4875-b4af-902ea9e1d5a3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:38.529317 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14829824-afbe-4875-b4af-902ea9e1d5a3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:38.530902 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14829824-afbe-4875-b4af-902ea9e1d5a3 service cinder] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:38.540949 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1353/2729] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 10:04:38 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:04:44.312122 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8dc6ee-9634-4a96-940b-f01823404822 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:44.312667 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8dc6ee-9634-4a96-940b-f01823404822 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:44.312667 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8dc6ee-9634-4a96-940b-f01823404822 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:44.343121 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-ae8dc6ee-9634-4a96-940b-f01823404822 None None] MFA Rules not processed for user `2cb599e8e13f4932bc27ab2770b8963c`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:04:44.344579 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-ae8dc6ee-9634-4a96-940b-f01823404822 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:44.427402 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1377/2730] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 10:04:44 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:04:44.439044 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14829824-afbe-4875-b4af-902ea9e1d5a3 service cinder] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:44.440590 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-14829824-afbe-4875-b4af-902ea9e1d5a3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:44.500891 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14829824-afbe-4875-b4af-902ea9e1d5a3 service cinder] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:44.504941 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b60683-ea31-4447-90ce-2b36a774fa53 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea2666d430a646d386104addf1f42888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:44.506074 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b60683-ea31-4447-90ce-2b36a774fa53 service glance] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:44.506361 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b60683-ea31-4447-90ce-2b36a774fa53 service glance] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:44.506636 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b60683-ea31-4447-90ce-2b36a774fa53 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:44.566466 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b60683-ea31-4447-90ce-2b36a774fa53 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:44.568461 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b60683-ea31-4447-90ce-2b36a774fa53 service glance] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:44.583743 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1354/2731] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 10:04:44 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:04:44.867372 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b109e5c3-391d-4f5d-be93-c25f2173f962 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:44.867691 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b109e5c3-391d-4f5d-be93-c25f2173f962 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:44.867968 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b109e5c3-391d-4f5d-be93-c25f2173f962 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:44.989263 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-b109e5c3-391d-4f5d-be93-c25f2173f962 None None] MFA Rules not processed for user `0489462287fc4aea8e8e720af24e1ef8`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:04:44.991390 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-b109e5c3-391d-4f5d-be93-c25f2173f962 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:45.073384 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1378/2732] 10.222.0.16 () {58 vars in 1009 bytes} [Wed Aug 24 10:04:44 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:04:45.214882 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b60683-ea31-4447-90ce-2b36a774fa53 service glance] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:45.216363 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-58b60683-ea31-4447-90ce-2b36a774fa53 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:45.266979 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:04:45.318945 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b60683-ea31-4447-90ce-2b36a774fa53 service glance] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:45.324213 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590c8db2-5705-4f73-9b2e-ea938ca618db service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdd5f0d67fe54e9fae6f4af644988d93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:45.324834 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590c8db2-5705-4f73-9b2e-ea938ca618db service swift] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:45.325120 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590c8db2-5705-4f73-9b2e-ea938ca618db service swift] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:45.325400 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590c8db2-5705-4f73-9b2e-ea938ca618db service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:45.338021 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590c8db2-5705-4f73-9b2e-ea938ca618db service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:45.339753 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590c8db2-5705-4f73-9b2e-ea938ca618db service swift] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:45.348483 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1355/2733] 10.222.0.16 () {60 vars in 1384 bytes} [Wed Aug 24 10:04:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 10:04:46.013913 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b109e5c3-391d-4f5d-be93-c25f2173f962 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:46.015487 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-b109e5c3-391d-4f5d-be93-c25f2173f962 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:46.086616 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:04:46.105435 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b109e5c3-391d-4f5d-be93-c25f2173f962 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:46.113243 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e956cd1-e942-4b53-a57f-556c5ed882df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:46.114011 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e956cd1-e942-4b53-a57f-556c5ed882df admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:46.114368 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e956cd1-e942-4b53-a57f-556c5ed882df admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:46.114711 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e956cd1-e942-4b53-a57f-556c5ed882df admin admin] PATH_INFO: `/v3/users/a2c670e02fa64a1a9390cc1908c5545a` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:46.115993 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e956cd1-e942-4b53-a57f-556c5ed882df admin admin] RBAC: Authorizing `identity:delete_user(user_id=a2c670e02fa64a1a9390cc1908c5545a)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:46.117972 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e956cd1-e942-4b53-a57f-556c5ed882df admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:46.289987 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-2e956cd1-e942-4b53-a57f-556c5ed882df admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a2c670e02fa64a1a9390cc1908c5545a'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:46.291199 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-2e956cd1-e942-4b53-a57f-556c5ed882df admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a2c670e02fa64a1a9390cc1908c5545a'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:46.358016 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-2e956cd1-e942-4b53-a57f-556c5ed882df admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a2c670e02fa64a1a9390cc1908c5545a'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:46.358719 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-2e956cd1-e942-4b53-a57f-556c5ed882df admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a2c670e02fa64a1a9390cc1908c5545a'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:46.368423 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-2e956cd1-e942-4b53-a57f-556c5ed882df admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a2c670e02fa64a1a9390cc1908c5545a'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:46.401736 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fccf69-e204-4fd2-ab55-2c85630aa6e7 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:46.402307 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fccf69-e204-4fd2-ab55-2c85630aa6e7 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:46.402307 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fccf69-e204-4fd2-ab55-2c85630aa6e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:46.405060 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1379/2734] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 24 10:04:46 2022] DELETE /identity/v3/users/a2c670e02fa64a1a9390cc1908c5545a => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:04:46.423465 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-79fccf69-e204-4fd2-ab55-2c85630aa6e7 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:04:46.425317 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-79fccf69-e204-4fd2-ab55-2c85630aa6e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:46.506561 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1356/2735] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 10:04:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:04:46.516610 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e956cd1-e942-4b53-a57f-556c5ed882df admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:46.518283 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-2e956cd1-e942-4b53-a57f-556c5ed882df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:46.584350 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e956cd1-e942-4b53-a57f-556c5ed882df admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:46.587722 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3bfe8f-5119-4b7c-bbd7-2b600afbb3f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e5260e798404e8bb12b25ca8ea97052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:46.588349 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3bfe8f-5119-4b7c-bbd7-2b600afbb3f5 service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:46.588636 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3bfe8f-5119-4b7c-bbd7-2b600afbb3f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:46.588913 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3bfe8f-5119-4b7c-bbd7-2b600afbb3f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:46.600802 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3bfe8f-5119-4b7c-bbd7-2b600afbb3f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:46.602857 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3bfe8f-5119-4b7c-bbd7-2b600afbb3f5 service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:46.630591 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1380/2736] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 10:04:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:04:47.819042 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fccf69-e204-4fd2-ab55-2c85630aa6e7 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:47.823667 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-79fccf69-e204-4fd2-ab55-2c85630aa6e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:47.845166 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fccf69-e204-4fd2-ab55-2c85630aa6e7 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:47.864369 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d010a0f-3432-4988-b3f6-3dbd6df19249 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:47.873440 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d010a0f-3432-4988-b3f6-3dbd6df19249 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:47.873988 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d010a0f-3432-4988-b3f6-3dbd6df19249 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:47.874607 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d010a0f-3432-4988-b3f6-3dbd6df19249 admin admin] PATH_INFO: `/v3/projects/d8a1ebc1209246e6a4e472cf816dff88` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:47.876192 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d010a0f-3432-4988-b3f6-3dbd6df19249 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d8a1ebc1209246e6a4e472cf816dff88)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:47.878437 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d010a0f-3432-4988-b3f6-3dbd6df19249 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:48.039096 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-9d010a0f-3432-4988-b3f6-3dbd6df19249 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd8a1ebc1209246e6a4e472cf816dff88', 'request_id': 'req-9d010a0f-3432-4988-b3f6-3dbd6df19249'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:48.054620 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-9d010a0f-3432-4988-b3f6-3dbd6df19249 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd8a1ebc1209246e6a4e472cf816dff88', 'request_id': 'req-9d010a0f-3432-4988-b3f6-3dbd6df19249'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:48.061764 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-9d010a0f-3432-4988-b3f6-3dbd6df19249 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd8a1ebc1209246e6a4e472cf816dff88', 'request_id': 'req-9d010a0f-3432-4988-b3f6-3dbd6df19249'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:48.260456 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-9d010a0f-3432-4988-b3f6-3dbd6df19249 admin admin] The token cache is being invalidate because project d8a1ebc1209246e6a4e472cf816dff88 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:04:48.260746 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-9d010a0f-3432-4988-b3f6-3dbd6df19249 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:48.261510 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-9d010a0f-3432-4988-b3f6-3dbd6df19249 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:48.263167 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1357/2737] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 24 10:04:47 2022] DELETE /identity/v3/projects/d8a1ebc1209246e6a4e472cf816dff88 => generated 0 bytes in 445 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:04:48.267257 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3bfe8f-5119-4b7c-bbd7-2b600afbb3f5 service neutron] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:48.277455 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3bfe8f-5119-4b7c-bbd7-2b600afbb3f5 service neutron] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:48.357902 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25acd5df-61c8-4d1b-86c6-68480c9b6e31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:48.358504 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25acd5df-61c8-4d1b-86c6-68480c9b6e31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:48.358789 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25acd5df-61c8-4d1b-86c6-68480c9b6e31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:48.359065 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25acd5df-61c8-4d1b-86c6-68480c9b6e31 admin admin] PATH_INFO: `/v3/users/3e641bbb85744261abd1440a427c936c` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:48.360436 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25acd5df-61c8-4d1b-86c6-68480c9b6e31 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3e641bbb85744261abd1440a427c936c)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:48.362361 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25acd5df-61c8-4d1b-86c6-68480c9b6e31 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:48.382500 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:04:48.614904 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-25acd5df-61c8-4d1b-86c6-68480c9b6e31 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3e641bbb85744261abd1440a427c936c'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:48.615901 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-25acd5df-61c8-4d1b-86c6-68480c9b6e31 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3e641bbb85744261abd1440a427c936c'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:48.702070 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-25acd5df-61c8-4d1b-86c6-68480c9b6e31 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3e641bbb85744261abd1440a427c936c'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:48.702743 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-25acd5df-61c8-4d1b-86c6-68480c9b6e31 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3e641bbb85744261abd1440a427c936c'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:48.717259 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-25acd5df-61c8-4d1b-86c6-68480c9b6e31 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3e641bbb85744261abd1440a427c936c'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:48.751832 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1381/2738] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 24 10:04:48 2022] DELETE /identity/v3/users/3e641bbb85744261abd1440a427c936c => generated 0 bytes in 485 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:04:49.376509 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d010a0f-3432-4988-b3f6-3dbd6df19249 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:49.381432 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-9d010a0f-3432-4988-b3f6-3dbd6df19249 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:49.401375 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d010a0f-3432-4988-b3f6-3dbd6df19249 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:49.474174 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:04:49.503602 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705f0a86-cdee-4ad1-b45c-4ac97c8a22d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:49.504292 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705f0a86-cdee-4ad1-b45c-4ac97c8a22d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:49.504579 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705f0a86-cdee-4ad1-b45c-4ac97c8a22d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:49.504858 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705f0a86-cdee-4ad1-b45c-4ac97c8a22d5 admin admin] PATH_INFO: `/v3/projects/7e3da9bbdf1144c48b4a43fff59200b8` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:49.506666 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705f0a86-cdee-4ad1-b45c-4ac97c8a22d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e3da9bbdf1144c48b4a43fff59200b8)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:49.510842 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705f0a86-cdee-4ad1-b45c-4ac97c8a22d5 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:49.623694 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-705f0a86-cdee-4ad1-b45c-4ac97c8a22d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e3da9bbdf1144c48b4a43fff59200b8', 'request_id': 'req-705f0a86-cdee-4ad1-b45c-4ac97c8a22d5'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:49.630279 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-705f0a86-cdee-4ad1-b45c-4ac97c8a22d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e3da9bbdf1144c48b4a43fff59200b8', 'request_id': 'req-705f0a86-cdee-4ad1-b45c-4ac97c8a22d5'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:49.637658 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-705f0a86-cdee-4ad1-b45c-4ac97c8a22d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e3da9bbdf1144c48b4a43fff59200b8', 'request_id': 'req-705f0a86-cdee-4ad1-b45c-4ac97c8a22d5'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:49.726234 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-705f0a86-cdee-4ad1-b45c-4ac97c8a22d5 admin admin] The token cache is being invalidate because project 7e3da9bbdf1144c48b4a43fff59200b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:04:49.726532 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-705f0a86-cdee-4ad1-b45c-4ac97c8a22d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:49.727189 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-705f0a86-cdee-4ad1-b45c-4ac97c8a22d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:49.728941 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1358/2739] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 24 10:04:49 2022] DELETE /identity/v3/projects/7e3da9bbdf1144c48b4a43fff59200b8 => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:04:49.735766 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6795291-1e4e-4379-a355-60379cdb1de6 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:49.736321 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6795291-1e4e-4379-a355-60379cdb1de6 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:49.736321 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6795291-1e4e-4379-a355-60379cdb1de6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:49.758875 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-d6795291-1e4e-4379-a355-60379cdb1de6 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:04:49.760485 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-d6795291-1e4e-4379-a355-60379cdb1de6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:49.830838 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1382/2740] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 10:04:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:04:49.840703 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8198f57-ec21-4e73-8da4-eaa9918852dd None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:49.841038 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8198f57-ec21-4e73-8da4-eaa9918852dd None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:49.841257 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8198f57-ec21-4e73-8da4-eaa9918852dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:49.862771 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-e8198f57-ec21-4e73-8da4-eaa9918852dd None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:04:49.864195 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-e8198f57-ec21-4e73-8da4-eaa9918852dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:49.873533 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1359/2741] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 10:04:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:04:49.877633 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6795291-1e4e-4379-a355-60379cdb1de6 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:49.887954 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6795291-1e4e-4379-a355-60379cdb1de6 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:49.892828 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b4ab39-7220-415d-b55e-96adcd6cc854 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:49.893516 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b4ab39-7220-415d-b55e-96adcd6cc854 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:49.893810 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b4ab39-7220-415d-b55e-96adcd6cc854 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:49.894080 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b4ab39-7220-415d-b55e-96adcd6cc854 admin admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:49.894732 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b4ab39-7220-415d-b55e-96adcd6cc854 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:04:49.895039 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b4ab39-7220-415d-b55e-96adcd6cc854 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:49.896713 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b4ab39-7220-415d-b55e-96adcd6cc854 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:49.916526 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8198f57-ec21-4e73-8da4-eaa9918852dd None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:49.919773 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1383/2742] 10.222.0.16 () {58 vars in 1153 bytes} [Wed Aug 24 10:04:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:04:49.928799 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8198f57-ec21-4e73-8da4-eaa9918852dd None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:49.934511 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5505b8d2-f838-4ef1-bad2-67440bac0af0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:49.935121 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5505b8d2-f838-4ef1-bad2-67440bac0af0 admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:49.935403 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5505b8d2-f838-4ef1-bad2-67440bac0af0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:49.935716 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5505b8d2-f838-4ef1-bad2-67440bac0af0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:49.937009 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5505b8d2-f838-4ef1-bad2-67440bac0af0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:49.938899 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5505b8d2-f838-4ef1-bad2-67440bac0af0 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:50.055636 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1360/2743] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 24 10:04:49 2022] POST /identity/v3/projects => generated 359 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:04:50.058424 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b4ab39-7220-415d-b55e-96adcd6cc854 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:50.070221 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b4ab39-7220-415d-b55e-96adcd6cc854 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:50.131137 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bda8b3-8281-49af-bf4f-75174d09bb7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:50.133239 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bda8b3-8281-49af-bf4f-75174d09bb7c admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:50.133239 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bda8b3-8281-49af-bf4f-75174d09bb7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:50.133455 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bda8b3-8281-49af-bf4f-75174d09bb7c admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:50.133954 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36bda8b3-8281-49af-bf4f-75174d09bb7c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:50.135915 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36bda8b3-8281-49af-bf4f-75174d09bb7c admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:50.246113 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1384/2744] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 24 10:04:50 2022] POST /identity/v3/users => generated 280 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:04:50.252875 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5505b8d2-f838-4ef1-bad2-67440bac0af0 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:50.264042 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5505b8d2-f838-4ef1-bad2-67440bac0af0 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:50.270590 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fe2511-fd3c-4c1e-a745-0cf86b016bd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:50.271925 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fe2511-fd3c-4c1e-a745-0cf86b016bd5 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:50.278131 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fe2511-fd3c-4c1e-a745-0cf86b016bd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:50.278131 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fe2511-fd3c-4c1e-a745-0cf86b016bd5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:50.278131 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fe2511-fd3c-4c1e-a745-0cf86b016bd5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:50.278131 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fe2511-fd3c-4c1e-a745-0cf86b016bd5 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:50.286295 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1361/2745] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 24 10:04:50 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:04:50.290925 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bda8b3-8281-49af-bf4f-75174d09bb7c admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:50.308142 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bda8b3-8281-49af-bf4f-75174d09bb7c admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:50.314110 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43b0d82-e104-45e1-8f26-885ef83def9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:50.314825 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43b0d82-e104-45e1-8f26-885ef83def9e admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:50.315176 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43b0d82-e104-45e1-8f26-885ef83def9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:50.315523 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43b0d82-e104-45e1-8f26-885ef83def9e admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:50.316263 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43b0d82-e104-45e1-8f26-885ef83def9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:50.317975 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43b0d82-e104-45e1-8f26-885ef83def9e admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:50.319671 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:04:50.335835 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1385/2746] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 24 10:04:50 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:04:50.339718 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fe2511-fd3c-4c1e-a745-0cf86b016bd5 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:50.351751 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fe2511-fd3c-4c1e-a745-0cf86b016bd5 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:50.356942 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca24f523-d52a-41da-b39f-350267a61b1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:50.357801 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca24f523-d52a-41da-b39f-350267a61b1b admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:50.358243 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca24f523-d52a-41da-b39f-350267a61b1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:50.359311 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca24f523-d52a-41da-b39f-350267a61b1b admin admin] PATH_INFO: `/v3/projects/4cf0777b2c5141a88dd5045b7617cd3a/users/a61de49f9da7444c9ae0483f72d51f70/roles/136cdf78e00a426997dd4e37faff489b` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:50.377755 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca24f523-d52a-41da-b39f-350267a61b1b admin admin] RBAC: Authorizing `identity:create_grant(project_id=4cf0777b2c5141a88dd5045b7617cd3a, user_id=a61de49f9da7444c9ae0483f72d51f70, role_id=136cdf78e00a426997dd4e37faff489b)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:50.381439 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca24f523-d52a-41da-b39f-350267a61b1b admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:50.441056 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1362/2747] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 24 10:04:50 2022] PUT /identity/v3/projects/4cf0777b2c5141a88dd5045b7617cd3a/users/a61de49f9da7444c9ae0483f72d51f70/roles/136cdf78e00a426997dd4e37faff489b => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 10:04:50.452936 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0229983a-67ca-4a57-84c2-d2ce7e1a1d41 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:50.453223 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0229983a-67ca-4a57-84c2-d2ce7e1a1d41 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:50.453503 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0229983a-67ca-4a57-84c2-d2ce7e1a1d41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:50.488122 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-0229983a-67ca-4a57-84c2-d2ce7e1a1d41 None None] MFA Rules not processed for user `a61de49f9da7444c9ae0483f72d51f70`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:04:50.489453 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-0229983a-67ca-4a57-84c2-d2ce7e1a1d41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:50.571265 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1386/2748] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 10:04:50 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:04:50.617601 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca24f523-d52a-41da-b39f-350267a61b1b admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:50.619816 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-ca24f523-d52a-41da-b39f-350267a61b1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:50.700280 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca24f523-d52a-41da-b39f-350267a61b1b admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:50.704179 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bc0e87-7382-47b9-9963-c2677de5c92f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c32b83577c46488906fe409a790b24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:50.704874 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bc0e87-7382-47b9-9963-c2677de5c92f service cinder] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:50.705240 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bc0e87-7382-47b9-9963-c2677de5c92f service cinder] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:50.705584 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bc0e87-7382-47b9-9963-c2677de5c92f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:50.718038 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bc0e87-7382-47b9-9963-c2677de5c92f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:50.720117 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bc0e87-7382-47b9-9963-c2677de5c92f service cinder] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:50.731350 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1363/2749] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 10:04:50 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:04:50.750418 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0229983a-67ca-4a57-84c2-d2ce7e1a1d41 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:50.828608 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0229983a-67ca-4a57-84c2-d2ce7e1a1d41 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:50.838087 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9af00e-6690-44df-9abd-1e9410df00bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:50.839493 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9af00e-6690-44df-9abd-1e9410df00bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:50.839965 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9af00e-6690-44df-9abd-1e9410df00bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:50.840841 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9af00e-6690-44df-9abd-1e9410df00bd admin admin] PATH_INFO: `/v3/users/a61de49f9da7444c9ae0483f72d51f70` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:50.842469 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9af00e-6690-44df-9abd-1e9410df00bd admin admin] RBAC: Authorizing `identity:delete_user(user_id=a61de49f9da7444c9ae0483f72d51f70)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:50.848439 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9af00e-6690-44df-9abd-1e9410df00bd admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:51.516906 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-eb9af00e-6690-44df-9abd-1e9410df00bd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a61de49f9da7444c9ae0483f72d51f70'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:51.517614 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-eb9af00e-6690-44df-9abd-1e9410df00bd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a61de49f9da7444c9ae0483f72d51f70'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:51.673759 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-eb9af00e-6690-44df-9abd-1e9410df00bd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a61de49f9da7444c9ae0483f72d51f70'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:51.674390 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-eb9af00e-6690-44df-9abd-1e9410df00bd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a61de49f9da7444c9ae0483f72d51f70'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:51.682745 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-eb9af00e-6690-44df-9abd-1e9410df00bd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a61de49f9da7444c9ae0483f72d51f70'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:51.701183 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1387/2750] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 24 10:04:50 2022] DELETE /identity/v3/users/a61de49f9da7444c9ae0483f72d51f70 => generated 0 bytes in 952 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:04:51.705686 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147b2710-7053-46c4-88f0-7ea9e85a6dec None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:51.706146 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147b2710-7053-46c4-88f0-7ea9e85a6dec None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:51.706547 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147b2710-7053-46c4-88f0-7ea9e85a6dec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:51.714586 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:04:51.725112 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-147b2710-7053-46c4-88f0-7ea9e85a6dec None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:04:51.726326 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-147b2710-7053-46c4-88f0-7ea9e85a6dec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:51.801671 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1364/2751] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 10:04:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:04:51.809288 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9af00e-6690-44df-9abd-1e9410df00bd admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:51.814146 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-eb9af00e-6690-44df-9abd-1e9410df00bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:51.818598 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:04:51.886484 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9af00e-6690-44df-9abd-1e9410df00bd admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:51.890360 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce96d91-06c6-438c-8829-212f4d98cb07 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e5260e798404e8bb12b25ca8ea97052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:51.890981 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce96d91-06c6-438c-8829-212f4d98cb07 service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:51.891263 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce96d91-06c6-438c-8829-212f4d98cb07 service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:51.891536 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce96d91-06c6-438c-8829-212f4d98cb07 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:51.901565 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce96d91-06c6-438c-8829-212f4d98cb07 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:51.903011 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce96d91-06c6-438c-8829-212f4d98cb07 service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:51.913904 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1388/2752] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 10:04:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:04:52.386321 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147b2710-7053-46c4-88f0-7ea9e85a6dec None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:52.387558 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-147b2710-7053-46c4-88f0-7ea9e85a6dec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:52.409176 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147b2710-7053-46c4-88f0-7ea9e85a6dec None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:52.411290 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee9d886-f334-4263-9322-87d954678472 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:52.412733 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee9d886-f334-4263-9322-87d954678472 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:52.413334 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee9d886-f334-4263-9322-87d954678472 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:52.413784 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee9d886-f334-4263-9322-87d954678472 admin admin] PATH_INFO: `/v3/projects/4cf0777b2c5141a88dd5045b7617cd3a` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:52.415122 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee9d886-f334-4263-9322-87d954678472 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4cf0777b2c5141a88dd5045b7617cd3a)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:52.418371 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee9d886-f334-4263-9322-87d954678472 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:52.593971 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-6ee9d886-f334-4263-9322-87d954678472 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4cf0777b2c5141a88dd5045b7617cd3a', 'request_id': 'req-6ee9d886-f334-4263-9322-87d954678472'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:52.602292 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-6ee9d886-f334-4263-9322-87d954678472 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4cf0777b2c5141a88dd5045b7617cd3a', 'request_id': 'req-6ee9d886-f334-4263-9322-87d954678472'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:52.609349 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-6ee9d886-f334-4263-9322-87d954678472 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4cf0777b2c5141a88dd5045b7617cd3a', 'request_id': 'req-6ee9d886-f334-4263-9322-87d954678472'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:52.692830 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-6ee9d886-f334-4263-9322-87d954678472 admin admin] The token cache is being invalidate because project 4cf0777b2c5141a88dd5045b7617cd3a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:04:52.693136 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-6ee9d886-f334-4263-9322-87d954678472 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:52.693634 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-6ee9d886-f334-4263-9322-87d954678472 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:52.695993 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1365/2753] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 24 10:04:52 2022] DELETE /identity/v3/projects/4cf0777b2c5141a88dd5045b7617cd3a => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:04:52.701080 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387bece9-8224-4064-a741-a263515c8553 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:52.701488 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387bece9-8224-4064-a741-a263515c8553 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:52.701869 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387bece9-8224-4064-a741-a263515c8553 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:52.731543 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-387bece9-8224-4064-a741-a263515c8553 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:04:52.733125 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-387bece9-8224-4064-a741-a263515c8553 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:52.803229 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1389/2754] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 10:04:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:04:52.811104 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a440366b-e483-4038-904b-89ce51b76055 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:52.811675 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a440366b-e483-4038-904b-89ce51b76055 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:52.811824 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a440366b-e483-4038-904b-89ce51b76055 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:52.835079 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-a440366b-e483-4038-904b-89ce51b76055 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:04:52.836353 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-a440366b-e483-4038-904b-89ce51b76055 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:52.844443 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1366/2755] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 10:04:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:04:52.848893 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387bece9-8224-4064-a741-a263515c8553 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:52.858875 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387bece9-8224-4064-a741-a263515c8553 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:52.864607 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383f2a85-d90b-4848-9cd1-d35a4c448b48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:52.865521 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383f2a85-d90b-4848-9cd1-d35a4c448b48 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:52.885904 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383f2a85-d90b-4848-9cd1-d35a4c448b48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:52.885904 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383f2a85-d90b-4848-9cd1-d35a4c448b48 admin admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:52.885904 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383f2a85-d90b-4848-9cd1-d35a4c448b48 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:04:52.885904 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383f2a85-d90b-4848-9cd1-d35a4c448b48 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:52.885904 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383f2a85-d90b-4848-9cd1-d35a4c448b48 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:52.885904 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1390/2756] 10.222.0.16 () {58 vars in 1153 bytes} [Wed Aug 24 10:04:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:04:52.890823 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a440366b-e483-4038-904b-89ce51b76055 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:52.900895 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a440366b-e483-4038-904b-89ce51b76055 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:52.906676 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1352fdb3-0666-43f7-bdd7-c6ba2f6f1ab6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:52.907297 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1352fdb3-0666-43f7-bdd7-c6ba2f6f1ab6 admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:52.908248 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1352fdb3-0666-43f7-bdd7-c6ba2f6f1ab6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:52.908538 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1352fdb3-0666-43f7-bdd7-c6ba2f6f1ab6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:52.909189 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1352fdb3-0666-43f7-bdd7-c6ba2f6f1ab6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:52.911135 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1352fdb3-0666-43f7-bdd7-c6ba2f6f1ab6 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:53.009329 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1367/2757] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 24 10:04:52 2022] POST /identity/v3/projects => generated 375 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:04:53.013606 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383f2a85-d90b-4848-9cd1-d35a4c448b48 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:53.026138 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383f2a85-d90b-4848-9cd1-d35a4c448b48 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:53.091127 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d40809-71ae-4483-b5d2-b0e5e1d1b4cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:53.092061 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d40809-71ae-4483-b5d2-b0e5e1d1b4cb admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:53.092392 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d40809-71ae-4483-b5d2-b0e5e1d1b4cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:53.092771 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d40809-71ae-4483-b5d2-b0e5e1d1b4cb admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:53.093449 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d40809-71ae-4483-b5d2-b0e5e1d1b4cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:53.095424 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d40809-71ae-4483-b5d2-b0e5e1d1b4cb admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:53.178084 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1391/2758] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 24 10:04:53 2022] POST /identity/v3/users => generated 288 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 10:04:53.184356 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1352fdb3-0666-43f7-bdd7-c6ba2f6f1ab6 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:53.196757 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1352fdb3-0666-43f7-bdd7-c6ba2f6f1ab6 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:53.204453 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70320ef3-4b3f-4417-a696-58acf875b21f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:53.205099 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70320ef3-4b3f-4417-a696-58acf875b21f admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:53.205376 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70320ef3-4b3f-4417-a696-58acf875b21f admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:53.205662 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70320ef3-4b3f-4417-a696-58acf875b21f admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:53.206297 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70320ef3-4b3f-4417-a696-58acf875b21f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:53.208437 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70320ef3-4b3f-4417-a696-58acf875b21f admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:53.224664 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1368/2759] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 24 10:04:53 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:04:53.225699 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d40809-71ae-4483-b5d2-b0e5e1d1b4cb admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:53.236246 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d40809-71ae-4483-b5d2-b0e5e1d1b4cb admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:53.244198 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39afb1dc-7030-4278-8b2e-c397f9bb3199 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:53.245022 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39afb1dc-7030-4278-8b2e-c397f9bb3199 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:53.245479 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39afb1dc-7030-4278-8b2e-c397f9bb3199 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:53.245894 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39afb1dc-7030-4278-8b2e-c397f9bb3199 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:53.246614 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39afb1dc-7030-4278-8b2e-c397f9bb3199 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:53.248416 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39afb1dc-7030-4278-8b2e-c397f9bb3199 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:53.263943 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1392/2760] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 24 10:04:53 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:04:53.267986 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70320ef3-4b3f-4417-a696-58acf875b21f admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:53.281175 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70320ef3-4b3f-4417-a696-58acf875b21f admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:53.286179 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9695463-10ab-4860-8a3b-9f8d32c8c4d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:53.286911 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9695463-10ab-4860-8a3b-9f8d32c8c4d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:53.287187 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9695463-10ab-4860-8a3b-9f8d32c8c4d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:53.287457 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9695463-10ab-4860-8a3b-9f8d32c8c4d1 admin admin] PATH_INFO: `/v3/projects/3d902f7914f649e0b534a24e469e09c4/users/00cea9a8c4924fc18bb0e94d2adbb92b/roles/136cdf78e00a426997dd4e37faff489b` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:53.307510 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9695463-10ab-4860-8a3b-9f8d32c8c4d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d902f7914f649e0b534a24e469e09c4, user_id=00cea9a8c4924fc18bb0e94d2adbb92b, role_id=136cdf78e00a426997dd4e37faff489b)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:53.310311 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9695463-10ab-4860-8a3b-9f8d32c8c4d1 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:53.378054 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1369/2761] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 24 10:04:53 2022] PUT /identity/v3/projects/3d902f7914f649e0b534a24e469e09c4/users/00cea9a8c4924fc18bb0e94d2adbb92b/roles/136cdf78e00a426997dd4e37faff489b => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 10:04:53.379878 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40df5d70-d565-4422-90c7-f513dfd3faf2 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:53.380158 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40df5d70-d565-4422-90c7-f513dfd3faf2 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:53.381161 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40df5d70-d565-4422-90c7-f513dfd3faf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:53.409119 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:04:53.413274 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-40df5d70-d565-4422-90c7-f513dfd3faf2 None None] MFA Rules not processed for user `00cea9a8c4924fc18bb0e94d2adbb92b`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:04:53.414743 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-40df5d70-d565-4422-90c7-f513dfd3faf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:53.492081 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1393/2762] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 10:04:53 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:04:53.502874 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9695463-10ab-4860-8a3b-9f8d32c8c4d1 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:53.504774 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-f9695463-10ab-4860-8a3b-9f8d32c8c4d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:53.537596 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:04:53.587470 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9695463-10ab-4860-8a3b-9f8d32c8c4d1 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:53.591633 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35139d30-c16f-48cf-a378-dcb09d44cef3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c32b83577c46488906fe409a790b24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:53.592457 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35139d30-c16f-48cf-a378-dcb09d44cef3 service cinder] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:53.593010 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35139d30-c16f-48cf-a378-dcb09d44cef3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:53.593483 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35139d30-c16f-48cf-a378-dcb09d44cef3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:53.605881 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35139d30-c16f-48cf-a378-dcb09d44cef3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:53.608120 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35139d30-c16f-48cf-a378-dcb09d44cef3 service cinder] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:53.618697 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1370/2763] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 10:04:53 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:04:56.397585 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5306e89f-b3de-46a0-a9c7-e0ff29e7ce20 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:56.402782 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5306e89f-b3de-46a0-a9c7-e0ff29e7ce20 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:56.402782 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5306e89f-b3de-46a0-a9c7-e0ff29e7ce20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:56.416568 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-5306e89f-b3de-46a0-a9c7-e0ff29e7ce20 None None] MFA Rules not processed for user `ea68de414dcd4ff79bfd0a1305c58f63`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:04:56.417708 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-5306e89f-b3de-46a0-a9c7-e0ff29e7ce20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:56.476039 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1394/2764] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 10:04:56 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:04:56.480950 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b126155-9982-4fad-99fd-0089a9754631 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:56.481435 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b126155-9982-4fad-99fd-0089a9754631 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:56.481543 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b126155-9982-4fad-99fd-0089a9754631 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:56.501441 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-5b126155-9982-4fad-99fd-0089a9754631 None None] MFA Rules not processed for user `00cea9a8c4924fc18bb0e94d2adbb92b`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:04:56.502654 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-5b126155-9982-4fad-99fd-0089a9754631 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:56.510615 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1371/2765] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 10:04:56 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:04:56.514485 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5306e89f-b3de-46a0-a9c7-e0ff29e7ce20 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:56.515991 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-5306e89f-b3de-46a0-a9c7-e0ff29e7ce20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:56.547712 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b126155-9982-4fad-99fd-0089a9754631 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:56.549162 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-5b126155-9982-4fad-99fd-0089a9754631 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:56.576340 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5306e89f-b3de-46a0-a9c7-e0ff29e7ce20 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:56.580030 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7978c744-114e-4637-b1ac-93d5580b854d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea2666d430a646d386104addf1f42888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:56.586365 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7978c744-114e-4637-b1ac-93d5580b854d service glance] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:56.586365 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7978c744-114e-4637-b1ac-93d5580b854d service glance] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:56.586365 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7978c744-114e-4637-b1ac-93d5580b854d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:56.586365 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-7978c744-114e-4637-b1ac-93d5580b854d service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:56.608611 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b126155-9982-4fad-99fd-0089a9754631 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:56.612409 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc57aec-6c9c-4e40-9297-c916e7dd3917 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea2666d430a646d386104addf1f42888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:56.613016 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc57aec-6c9c-4e40-9297-c916e7dd3917 service glance] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:56.613296 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc57aec-6c9c-4e40-9297-c916e7dd3917 service glance] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:56.613570 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc57aec-6c9c-4e40-9297-c916e7dd3917 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:56.615894 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-4cc57aec-6c9c-4e40-9297-c916e7dd3917 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:56.640175 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=100 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:04:56.654613 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7978c744-114e-4637-b1ac-93d5580b854d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:56.656091 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7978c744-114e-4637-b1ac-93d5580b854d service glance] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:56.669333 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1395/2766] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 10:04:56 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:04:56.689211 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc57aec-6c9c-4e40-9297-c916e7dd3917 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:56.691019 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc57aec-6c9c-4e40-9297-c916e7dd3917 service glance] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:56.702563 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1372/2767] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 10:04:56 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 10:04:56.893590 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7978c744-114e-4637-b1ac-93d5580b854d service glance] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:56.895021 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-7978c744-114e-4637-b1ac-93d5580b854d service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:56.952057 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7978c744-114e-4637-b1ac-93d5580b854d service glance] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:56.955544 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4cd6d4-2b37-4378-a6b6-d90cd85b8620 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e5260e798404e8bb12b25ca8ea97052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:56.956167 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4cd6d4-2b37-4378-a6b6-d90cd85b8620 service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:56.956446 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4cd6d4-2b37-4378-a6b6-d90cd85b8620 service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:56.956971 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4cd6d4-2b37-4378-a6b6-d90cd85b8620 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:56.959242 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-5e4cd6d4-2b37-4378-a6b6-d90cd85b8620 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:56.969619 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4cd6d4-2b37-4378-a6b6-d90cd85b8620 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:56.971043 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4cd6d4-2b37-4378-a6b6-d90cd85b8620 service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:56.979794 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1396/2768] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 10:04:56 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:05:00.729634 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b7fdae-38a8-4e09-a325-f7adbd43d30b None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:05:00.730599 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b7fdae-38a8-4e09-a325-f7adbd43d30b None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:05:00.730886 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b7fdae-38a8-4e09-a325-f7adbd43d30b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:05:00.760721 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-a6b7fdae-38a8-4e09-a325-f7adbd43d30b None None] MFA Rules not processed for user `0489462287fc4aea8e8e720af24e1ef8`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:05:00.762100 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-a6b7fdae-38a8-4e09-a325-f7adbd43d30b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:05:00.828683 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1373/2769] 10.222.0.16 () {58 vars in 1009 bytes} [Wed Aug 24 10:05:00 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:05:00.842641 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4cd6d4-2b37-4378-a6b6-d90cd85b8620 service neutron] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:05:00.845271 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-5e4cd6d4-2b37-4378-a6b6-d90cd85b8620 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:05:00.921919 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4cd6d4-2b37-4378-a6b6-d90cd85b8620 service neutron] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:05:00.926952 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8569b4f-0c09-4bc3-aae8-48ac102b4fd5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdd5f0d67fe54e9fae6f4af644988d93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:05:00.927606 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8569b4f-0c09-4bc3-aae8-48ac102b4fd5 service swift] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:05:00.927910 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8569b4f-0c09-4bc3-aae8-48ac102b4fd5 service swift] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:05:00.928185 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8569b4f-0c09-4bc3-aae8-48ac102b4fd5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:05:00.937080 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8569b4f-0c09-4bc3-aae8-48ac102b4fd5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:05:00.938627 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8569b4f-0c09-4bc3-aae8-48ac102b4fd5 service swift] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:05:00.947251 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1397/2770] 10.222.0.16 () {60 vars in 1384 bytes} [Wed Aug 24 10:05:00 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 10:05:10.820781 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=97 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:05:10.944031 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=63 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:05:15.118462 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b7fdae-38a8-4e09-a325-f7adbd43d30b None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:05:15.120646 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-a6b7fdae-38a8-4e09-a325-f7adbd43d30b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:05:15.140659 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b7fdae-38a8-4e09-a325-f7adbd43d30b None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:05:15.147020 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd031092-9ede-4a38-8cc4-901e3c6d5ef2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f055f7dc51d43bda9b315ca67f11d58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:05:15.147677 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd031092-9ede-4a38-8cc4-901e3c6d5ef2 service nova] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:05:15.148085 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd031092-9ede-4a38-8cc4-901e3c6d5ef2 service nova] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:05:15.148377 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd031092-9ede-4a38-8cc4-901e3c6d5ef2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:05:15.150278 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-dd031092-9ede-4a38-8cc4-901e3c6d5ef2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:05:15.165045 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd031092-9ede-4a38-8cc4-901e3c6d5ef2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:05:15.176230 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd031092-9ede-4a38-8cc4-901e3c6d5ef2 service nova] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:05:15.177668 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1374/2771] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 10:05:15 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:05:25.173180 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=6 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:05:32.113964 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df808f1-cbad-4576-a29a-0ce07dc1515b None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:05:32.114482 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df808f1-cbad-4576-a29a-0ce07dc1515b None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:05:32.114601 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df808f1-cbad-4576-a29a-0ce07dc1515b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:05:32.133195 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-1df808f1-cbad-4576-a29a-0ce07dc1515b None None] MFA Rules not processed for user `0489462287fc4aea8e8e720af24e1ef8`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:05:32.134386 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-1df808f1-cbad-4576-a29a-0ce07dc1515b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:05:32.142569 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1398/2772] 10.222.0.16 () {58 vars in 1009 bytes} [Wed Aug 24 10:05:32 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:05:32.154890 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd031092-9ede-4a38-8cc4-901e3c6d5ef2 service nova] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:05:32.167031 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd031092-9ede-4a38-8cc4-901e3c6d5ef2 service nova] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:05:32.172308 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ce8525-05c1-41a9-843e-4a7e5cc04184 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdd5f0d67fe54e9fae6f4af644988d93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:05:32.172929 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ce8525-05c1-41a9-843e-4a7e5cc04184 service swift] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:05:32.173217 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ce8525-05c1-41a9-843e-4a7e5cc04184 service swift] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:05:32.173492 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ce8525-05c1-41a9-843e-4a7e5cc04184 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:05:32.182167 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ce8525-05c1-41a9-843e-4a7e5cc04184 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:05:32.183770 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ce8525-05c1-41a9-843e-4a7e5cc04184 service swift] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:05:32.192118 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1375/2773] 10.222.0.16 () {60 vars in 1384 bytes} [Wed Aug 24 10:05:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 10:05:33.147335 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349bdbac-af97-4e35-af6e-2a322159f3df None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:05:33.147796 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349bdbac-af97-4e35-af6e-2a322159f3df None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:05:33.147929 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349bdbac-af97-4e35-af6e-2a322159f3df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:05:33.176275 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-349bdbac-af97-4e35-af6e-2a322159f3df None None] MFA Rules not processed for user `0489462287fc4aea8e8e720af24e1ef8`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:05:33.177457 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-349bdbac-af97-4e35-af6e-2a322159f3df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:05:33.204534 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1399/2774] 10.222.0.16 () {58 vars in 967 bytes} [Wed Aug 24 10:05:33 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:05:33.217661 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ce8525-05c1-41a9-843e-4a7e5cc04184 service swift] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:05:33.252761 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ce8525-05c1-41a9-843e-4a7e5cc04184 service swift] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:05:33.258705 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cca71b1-6904-4ba0-b753-ba6a12abedd2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdd5f0d67fe54e9fae6f4af644988d93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:05:33.259314 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cca71b1-6904-4ba0-b753-ba6a12abedd2 service swift] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:05:33.259605 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cca71b1-6904-4ba0-b753-ba6a12abedd2 service swift] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:05:33.259895 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cca71b1-6904-4ba0-b753-ba6a12abedd2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:05:33.267995 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cca71b1-6904-4ba0-b753-ba6a12abedd2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:05:33.269556 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cca71b1-6904-4ba0-b753-ba6a12abedd2 service swift] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:05:33.278160 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1376/2775] 10.222.0.16 () {60 vars in 1384 bytes} [Wed Aug 24 10:05:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 10:05:33.965757 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b0d0be-031e-4e93-bc0a-384669db5349 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:05:33.966036 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b0d0be-031e-4e93-bc0a-384669db5349 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:05:33.966304 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b0d0be-031e-4e93-bc0a-384669db5349 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:05:33.985998 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-07b0d0be-031e-4e93-bc0a-384669db5349 None None] MFA Rules not processed for user `0489462287fc4aea8e8e720af24e1ef8`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:05:33.987198 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-07b0d0be-031e-4e93-bc0a-384669db5349 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:05:33.995065 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1400/2776] 10.222.0.16 () {58 vars in 1009 bytes} [Wed Aug 24 10:05:33 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:05:34.006789 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cca71b1-6904-4ba0-b753-ba6a12abedd2 service swift] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:05:34.017745 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cca71b1-6904-4ba0-b753-ba6a12abedd2 service swift] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:05:34.023069 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26975fa6-355f-4cc2-a003-3d8bb518b0d7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdd5f0d67fe54e9fae6f4af644988d93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:05:34.023744 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26975fa6-355f-4cc2-a003-3d8bb518b0d7 service swift] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:05:34.024042 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26975fa6-355f-4cc2-a003-3d8bb518b0d7 service swift] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:05:34.024340 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26975fa6-355f-4cc2-a003-3d8bb518b0d7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:05:34.033065 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26975fa6-355f-4cc2-a003-3d8bb518b0d7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:05:34.034644 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26975fa6-355f-4cc2-a003-3d8bb518b0d7 service swift] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:05:34.046854 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1377/2777] 10.222.0.16 () {60 vars in 1384 bytes} [Wed Aug 24 10:05:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 10:05:34.783267 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cb1a82-86c3-4c23-ad94-162e3a908504 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:05:34.783756 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cb1a82-86c3-4c23-ad94-162e3a908504 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:05:34.783847 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cb1a82-86c3-4c23-ad94-162e3a908504 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:05:34.804668 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-52cb1a82-86c3-4c23-ad94-162e3a908504 None None] MFA Rules not processed for user `0489462287fc4aea8e8e720af24e1ef8`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:05:34.805884 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-52cb1a82-86c3-4c23-ad94-162e3a908504 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:05:34.813972 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1401/2778] 10.222.0.16 () {58 vars in 967 bytes} [Wed Aug 24 10:05:34 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:05:34.831402 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26975fa6-355f-4cc2-a003-3d8bb518b0d7 service swift] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:05:34.843084 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26975fa6-355f-4cc2-a003-3d8bb518b0d7 service swift] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:05:34.848093 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998f2f1b-04a8-4df3-aab8-3ad46260038c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdd5f0d67fe54e9fae6f4af644988d93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:05:34.848695 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998f2f1b-04a8-4df3-aab8-3ad46260038c service swift] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:05:34.848978 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998f2f1b-04a8-4df3-aab8-3ad46260038c service swift] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:05:34.849288 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998f2f1b-04a8-4df3-aab8-3ad46260038c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:05:34.857461 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998f2f1b-04a8-4df3-aab8-3ad46260038c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:05:34.859025 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998f2f1b-04a8-4df3-aab8-3ad46260038c service swift] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:05:34.867499 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1378/2779] 10.222.0.16 () {60 vars in 1384 bytes} [Wed Aug 24 10:05:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 10:05:37.967947 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cb1a82-86c3-4c23-ad94-162e3a908504 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:05:38.037609 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cb1a82-86c3-4c23-ad94-162e3a908504 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:05:38.042199 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a073e10e-a885-406a-b7fe-8fac081ea3ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:05:38.042825 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a073e10e-a885-406a-b7fe-8fac081ea3ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:05:38.043103 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a073e10e-a885-406a-b7fe-8fac081ea3ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:05:38.043371 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a073e10e-a885-406a-b7fe-8fac081ea3ca admin admin] PATH_INFO: `/v3/users/00cea9a8c4924fc18bb0e94d2adbb92b` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:05:38.044870 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a073e10e-a885-406a-b7fe-8fac081ea3ca admin admin] RBAC: Authorizing `identity:delete_user(user_id=00cea9a8c4924fc18bb0e94d2adbb92b)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:05:38.046796 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a073e10e-a885-406a-b7fe-8fac081ea3ca admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:05:38.704693 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-a073e10e-a885-406a-b7fe-8fac081ea3ca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '00cea9a8c4924fc18bb0e94d2adbb92b'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:05:38.705414 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-a073e10e-a885-406a-b7fe-8fac081ea3ca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '00cea9a8c4924fc18bb0e94d2adbb92b'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:05:38.781016 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-a073e10e-a885-406a-b7fe-8fac081ea3ca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '00cea9a8c4924fc18bb0e94d2adbb92b'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:05:38.785251 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-a073e10e-a885-406a-b7fe-8fac081ea3ca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '00cea9a8c4924fc18bb0e94d2adbb92b'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:05:38.789695 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-a073e10e-a885-406a-b7fe-8fac081ea3ca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '00cea9a8c4924fc18bb0e94d2adbb92b'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:05:38.806825 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1402/2780] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 24 10:05:37 2022] DELETE /identity/v3/users/00cea9a8c4924fc18bb0e94d2adbb92b => generated 0 bytes in 840 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:05:38.814233 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4d27b3-5439-47df-9e64-79b174db5458 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:05:38.814233 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4d27b3-5439-47df-9e64-79b174db5458 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:05:38.814233 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4d27b3-5439-47df-9e64-79b174db5458 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:05:38.833052 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-0c4d27b3-5439-47df-9e64-79b174db5458 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:05:38.835162 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-0c4d27b3-5439-47df-9e64-79b174db5458 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:05:39.019919 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1379/2781] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 10:05:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:05:39.030300 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a073e10e-a885-406a-b7fe-8fac081ea3ca admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:05:39.031776 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-a073e10e-a885-406a-b7fe-8fac081ea3ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:05:39.059072 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:05:39.088506 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a073e10e-a885-406a-b7fe-8fac081ea3ca admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:05:39.092078 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a435c11-1632-4eaa-be16-2cd4907ca3e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e5260e798404e8bb12b25ca8ea97052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:05:39.092688 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a435c11-1632-4eaa-be16-2cd4907ca3e3 service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:05:39.092970 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a435c11-1632-4eaa-be16-2cd4907ca3e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:05:39.093244 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a435c11-1632-4eaa-be16-2cd4907ca3e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:05:39.102913 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a435c11-1632-4eaa-be16-2cd4907ca3e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:05:39.104362 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a435c11-1632-4eaa-be16-2cd4907ca3e3 service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:05:39.114052 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1403/2782] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 10:05:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:05:39.391205 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4d27b3-5439-47df-9e64-79b174db5458 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:05:39.398033 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-0c4d27b3-5439-47df-9e64-79b174db5458 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:05:39.404865 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4d27b3-5439-47df-9e64-79b174db5458 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:05:39.409762 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6110a251-83cd-4ac0-986a-c1c9d3260442 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:05:39.410387 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6110a251-83cd-4ac0-986a-c1c9d3260442 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:05:39.410662 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6110a251-83cd-4ac0-986a-c1c9d3260442 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:05:39.410960 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6110a251-83cd-4ac0-986a-c1c9d3260442 admin admin] PATH_INFO: `/v3/projects/3d902f7914f649e0b534a24e469e09c4` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:05:39.412786 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6110a251-83cd-4ac0-986a-c1c9d3260442 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3d902f7914f649e0b534a24e469e09c4)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:05:39.414667 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6110a251-83cd-4ac0-986a-c1c9d3260442 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:05:39.583075 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-6110a251-83cd-4ac0-986a-c1c9d3260442 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3d902f7914f649e0b534a24e469e09c4', 'request_id': 'req-6110a251-83cd-4ac0-986a-c1c9d3260442'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:05:39.590103 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-6110a251-83cd-4ac0-986a-c1c9d3260442 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3d902f7914f649e0b534a24e469e09c4', 'request_id': 'req-6110a251-83cd-4ac0-986a-c1c9d3260442'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:05:39.595810 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-6110a251-83cd-4ac0-986a-c1c9d3260442 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3d902f7914f649e0b534a24e469e09c4', 'request_id': 'req-6110a251-83cd-4ac0-986a-c1c9d3260442'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:05:39.670086 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-6110a251-83cd-4ac0-986a-c1c9d3260442 admin admin] The token cache is being invalidate because project 3d902f7914f649e0b534a24e469e09c4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:05:39.670284 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-6110a251-83cd-4ac0-986a-c1c9d3260442 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:05:39.670898 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-6110a251-83cd-4ac0-986a-c1c9d3260442 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:05:39.672463 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1380/2783] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 24 10:05:39 2022] DELETE /identity/v3/projects/3d902f7914f649e0b534a24e469e09c4 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:05:44.156197 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a435c11-1632-4eaa-be16-2cd4907ca3e3 service neutron] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:05:44.165413 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a435c11-1632-4eaa-be16-2cd4907ca3e3 service neutron] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:05:44.212801 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4754dd99-dfc9-462c-ba5f-651acc5af858 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e5260e798404e8bb12b25ca8ea97052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:05:44.213404 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4754dd99-dfc9-462c-ba5f-651acc5af858 service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:05:44.213707 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4754dd99-dfc9-462c-ba5f-651acc5af858 service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:05:44.213977 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4754dd99-dfc9-462c-ba5f-651acc5af858 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:05:44.215755 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-4754dd99-dfc9-462c-ba5f-651acc5af858 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:05:44.280831 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4754dd99-dfc9-462c-ba5f-651acc5af858 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:05:44.282258 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4754dd99-dfc9-462c-ba5f-651acc5af858 service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:05:44.291673 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1404/2784] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 10:05:44 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 10:05:49.666200 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:05:54.287860 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=65 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:07:29.057922 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6110a251-83cd-4ac0-986a-c1c9d3260442 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:07:29.059333 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-6110a251-83cd-4ac0-986a-c1c9d3260442 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:07:29.161723 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6110a251-83cd-4ac0-986a-c1c9d3260442 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:07:29.166821 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75235972-2ce6-415b-bafc-53c9e06828bf service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80cada22039d4f11935412ee578acd05', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:07:29.167426 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75235972-2ce6-415b-bafc-53c9e06828bf service placement] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:07:29.167750 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75235972-2ce6-415b-bafc-53c9e06828bf service placement] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:07:29.168025 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75235972-2ce6-415b-bafc-53c9e06828bf service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:07:29.169750 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-75235972-2ce6-415b-bafc-53c9e06828bf service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:07:29.180918 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75235972-2ce6-415b-bafc-53c9e06828bf service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:07:29.182464 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75235972-2ce6-415b-bafc-53c9e06828bf service placement] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:07:29.192141 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1381/2785] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 10:07:29 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:07:39.188000 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=44 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:08:01.800624 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4754dd99-dfc9-462c-ba5f-651acc5af858 service neutron] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:08:01.802045 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-4754dd99-dfc9-462c-ba5f-651acc5af858 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:08:01.869963 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4754dd99-dfc9-462c-ba5f-651acc5af858 service neutron] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:08:01.873613 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e74f1e9-0082-472d-bf2d-339464ca92e4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c32b83577c46488906fe409a790b24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:08:01.874212 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e74f1e9-0082-472d-bf2d-339464ca92e4 service cinder] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:08:01.874493 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e74f1e9-0082-472d-bf2d-339464ca92e4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:08:01.874767 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e74f1e9-0082-472d-bf2d-339464ca92e4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:08:01.877147 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-9e74f1e9-0082-472d-bf2d-339464ca92e4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:08:01.933485 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e74f1e9-0082-472d-bf2d-339464ca92e4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:08:01.934908 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e74f1e9-0082-472d-bf2d-339464ca92e4 service cinder] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:08:01.944570 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1405/2786] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 10:08:01 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:08:11.940430 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=52 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:08:44.462904 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c163f6-d0ce-4de2-941c-4414b87ffe20 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:08:44.463462 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c163f6-d0ce-4de2-941c-4414b87ffe20 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:08:44.463462 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c163f6-d0ce-4de2-941c-4414b87ffe20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:08:44.483730 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-05c163f6-d0ce-4de2-941c-4414b87ffe20 None None] MFA Rules not processed for user `0489462287fc4aea8e8e720af24e1ef8`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:08:44.484864 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-05c163f6-d0ce-4de2-941c-4414b87ffe20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:08:44.540510 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1382/2787] 10.222.0.16 () {58 vars in 967 bytes} [Wed Aug 24 10:08:44 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:08:44.701101 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e74f1e9-0082-472d-bf2d-339464ca92e4 service cinder] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:08:44.702518 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-9e74f1e9-0082-472d-bf2d-339464ca92e4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:08:44.772497 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e74f1e9-0082-472d-bf2d-339464ca92e4 service cinder] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:08:44.777898 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633ba373-4f67-479c-86c1-24beae21548a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdd5f0d67fe54e9fae6f4af644988d93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:08:44.778503 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633ba373-4f67-479c-86c1-24beae21548a service swift] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:08:44.778781 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633ba373-4f67-479c-86c1-24beae21548a service swift] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:08:44.779059 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633ba373-4f67-479c-86c1-24beae21548a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:08:44.788518 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633ba373-4f67-479c-86c1-24beae21548a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:08:44.790133 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633ba373-4f67-479c-86c1-24beae21548a service swift] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:08:44.798014 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1406/2788] 10.222.0.16 () {60 vars in 1384 bytes} [Wed Aug 24 10:08:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 10:08:49.184168 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c163f6-d0ce-4de2-941c-4414b87ffe20 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:08:49.186067 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-05c163f6-d0ce-4de2-941c-4414b87ffe20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:08:49.201492 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c163f6-d0ce-4de2-941c-4414b87ffe20 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:08:49.206539 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9213b8d6-7a1e-4a9e-9a63-3c543bdd64ba service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80cada22039d4f11935412ee578acd05', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:08:49.207286 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9213b8d6-7a1e-4a9e-9a63-3c543bdd64ba service placement] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:08:49.207713 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9213b8d6-7a1e-4a9e-9a63-3c543bdd64ba service placement] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:08:49.207990 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9213b8d6-7a1e-4a9e-9a63-3c543bdd64ba service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:08:49.210240 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-9213b8d6-7a1e-4a9e-9a63-3c543bdd64ba service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:08:49.223363 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9213b8d6-7a1e-4a9e-9a63-3c543bdd64ba service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:08:49.225567 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9213b8d6-7a1e-4a9e-9a63-3c543bdd64ba service placement] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:08:49.236710 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1383/2789] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 10:08:49 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:08:52.940759 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633ba373-4f67-479c-86c1-24beae21548a service swift] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:08:52.942246 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-633ba373-4f67-479c-86c1-24beae21548a service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:08:53.008504 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633ba373-4f67-479c-86c1-24beae21548a service swift] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:08:53.013066 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cc0a3f-90f7-4907-a06e-a26135e1720b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:08:53.013705 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cc0a3f-90f7-4907-a06e-a26135e1720b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:08:53.014013 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cc0a3f-90f7-4907-a06e-a26135e1720b admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:08:53.014286 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cc0a3f-90f7-4907-a06e-a26135e1720b admin admin] PATH_INFO: `/v3/users/ea68de414dcd4ff79bfd0a1305c58f63` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:08:53.015441 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cc0a3f-90f7-4907-a06e-a26135e1720b admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea68de414dcd4ff79bfd0a1305c58f63)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:08:53.017389 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cc0a3f-90f7-4907-a06e-a26135e1720b admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:08:53.173339 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-13cc0a3f-90f7-4907-a06e-a26135e1720b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea68de414dcd4ff79bfd0a1305c58f63'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:08:53.173999 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-13cc0a3f-90f7-4907-a06e-a26135e1720b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea68de414dcd4ff79bfd0a1305c58f63'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:08:53.242454 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-13cc0a3f-90f7-4907-a06e-a26135e1720b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea68de414dcd4ff79bfd0a1305c58f63'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:08:53.243019 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-13cc0a3f-90f7-4907-a06e-a26135e1720b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea68de414dcd4ff79bfd0a1305c58f63'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:08:53.250192 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-13cc0a3f-90f7-4907-a06e-a26135e1720b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea68de414dcd4ff79bfd0a1305c58f63'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:08:53.268494 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1407/2790] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 24 10:08:52 2022] DELETE /identity/v3/users/ea68de414dcd4ff79bfd0a1305c58f63 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:08:53.496116 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9213b8d6-7a1e-4a9e-9a63-3c543bdd64ba service placement] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:08:53.507792 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9213b8d6-7a1e-4a9e-9a63-3c543bdd64ba service placement] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:08:53.567028 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9761da-0491-45f0-93fd-73a3df017778 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:08:53.567835 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9761da-0491-45f0-93fd-73a3df017778 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:08:53.568660 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9761da-0491-45f0-93fd-73a3df017778 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:08:53.568916 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9761da-0491-45f0-93fd-73a3df017778 admin admin] PATH_INFO: `/v3/projects/428f4e988b10492c87dddc5a0c4f34fb` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:08:53.570019 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9761da-0491-45f0-93fd-73a3df017778 admin admin] RBAC: Authorizing `identity:delete_project(project_id=428f4e988b10492c87dddc5a0c4f34fb)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:08:53.571926 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9761da-0491-45f0-93fd-73a3df017778 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:08:53.701332 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-5b9761da-0491-45f0-93fd-73a3df017778 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '428f4e988b10492c87dddc5a0c4f34fb', 'request_id': 'req-5b9761da-0491-45f0-93fd-73a3df017778'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:08:53.707613 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-5b9761da-0491-45f0-93fd-73a3df017778 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '428f4e988b10492c87dddc5a0c4f34fb', 'request_id': 'req-5b9761da-0491-45f0-93fd-73a3df017778'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:08:53.713232 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-5b9761da-0491-45f0-93fd-73a3df017778 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '428f4e988b10492c87dddc5a0c4f34fb', 'request_id': 'req-5b9761da-0491-45f0-93fd-73a3df017778'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:08:53.779810 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-5b9761da-0491-45f0-93fd-73a3df017778 admin admin] The token cache is being invalidate because project 428f4e988b10492c87dddc5a0c4f34fb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:08:53.785547 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-5b9761da-0491-45f0-93fd-73a3df017778 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:08:53.785547 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-5b9761da-0491-45f0-93fd-73a3df017778 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:08:53.785547 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1384/2791] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 24 10:08:53 2022] DELETE /identity/v3/projects/428f4e988b10492c87dddc5a0c4f34fb => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:08:53.787371 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cc0a3f-90f7-4907-a06e-a26135e1720b admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:08:53.788814 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-13cc0a3f-90f7-4907-a06e-a26135e1720b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:08:53.801695 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cc0a3f-90f7-4907-a06e-a26135e1720b admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:08:53.806858 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:08:53.857243 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76aef556-a193-4410-a6ba-6329445db411 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:08:53.857869 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76aef556-a193-4410-a6ba-6329445db411 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:08:53.858148 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76aef556-a193-4410-a6ba-6329445db411 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:08:53.858421 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76aef556-a193-4410-a6ba-6329445db411 admin admin] PATH_INFO: `/v3/users/2cb599e8e13f4932bc27ab2770b8963c` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:08:53.859739 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76aef556-a193-4410-a6ba-6329445db411 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2cb599e8e13f4932bc27ab2770b8963c)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:08:53.862224 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76aef556-a193-4410-a6ba-6329445db411 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:08:54.094343 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-76aef556-a193-4410-a6ba-6329445db411 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2cb599e8e13f4932bc27ab2770b8963c'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:08:54.095028 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-76aef556-a193-4410-a6ba-6329445db411 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2cb599e8e13f4932bc27ab2770b8963c'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:08:54.179365 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-76aef556-a193-4410-a6ba-6329445db411 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2cb599e8e13f4932bc27ab2770b8963c'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:08:54.180000 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-76aef556-a193-4410-a6ba-6329445db411 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2cb599e8e13f4932bc27ab2770b8963c'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:08:54.186967 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-76aef556-a193-4410-a6ba-6329445db411 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2cb599e8e13f4932bc27ab2770b8963c'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:08:54.203853 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1408/2792] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 24 10:08:53 2022] DELETE /identity/v3/users/2cb599e8e13f4932bc27ab2770b8963c => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:08:54.441815 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9761da-0491-45f0-93fd-73a3df017778 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:08:54.443378 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-5b9761da-0491-45f0-93fd-73a3df017778 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:08:54.456738 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9761da-0491-45f0-93fd-73a3df017778 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:08:54.478134 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:08:54.515918 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d7b62b-5d7b-45c6-9b7f-4bac9e224675 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:08:54.516546 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d7b62b-5d7b-45c6-9b7f-4bac9e224675 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:08:54.516814 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d7b62b-5d7b-45c6-9b7f-4bac9e224675 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:08:54.517087 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d7b62b-5d7b-45c6-9b7f-4bac9e224675 admin admin] PATH_INFO: `/v3/projects/ef18494b41144715bd8791e8a3cab9ea` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:08:54.518206 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d7b62b-5d7b-45c6-9b7f-4bac9e224675 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ef18494b41144715bd8791e8a3cab9ea)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:08:54.520135 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d7b62b-5d7b-45c6-9b7f-4bac9e224675 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:08:54.688555 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-f3d7b62b-5d7b-45c6-9b7f-4bac9e224675 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ef18494b41144715bd8791e8a3cab9ea', 'request_id': 'req-f3d7b62b-5d7b-45c6-9b7f-4bac9e224675'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:08:54.695084 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-f3d7b62b-5d7b-45c6-9b7f-4bac9e224675 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ef18494b41144715bd8791e8a3cab9ea', 'request_id': 'req-f3d7b62b-5d7b-45c6-9b7f-4bac9e224675'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:08:54.700668 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-f3d7b62b-5d7b-45c6-9b7f-4bac9e224675 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ef18494b41144715bd8791e8a3cab9ea', 'request_id': 'req-f3d7b62b-5d7b-45c6-9b7f-4bac9e224675'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:08:54.795287 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-f3d7b62b-5d7b-45c6-9b7f-4bac9e224675 admin admin] The token cache is being invalidate because project ef18494b41144715bd8791e8a3cab9ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:08:54.795287 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-f3d7b62b-5d7b-45c6-9b7f-4bac9e224675 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:08:54.795287 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-f3d7b62b-5d7b-45c6-9b7f-4bac9e224675 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:08:54.795287 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1385/2793] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 24 10:08:54 2022] DELETE /identity/v3/projects/ef18494b41144715bd8791e8a3cab9ea => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:09:04.200422 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=49,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:09:04.787973 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=41,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:12:30.138235 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76aef556-a193-4410-a6ba-6329445db411 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:12:30.140295 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-76aef556-a193-4410-a6ba-6329445db411 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:12:30.208327 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76aef556-a193-4410-a6ba-6329445db411 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:12:30.269937 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c535d1d5-b016-4d77-89e3-c833d2131c05 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80cada22039d4f11935412ee578acd05', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:12:30.270555 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c535d1d5-b016-4d77-89e3-c833d2131c05 service placement] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:12:30.270839 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c535d1d5-b016-4d77-89e3-c833d2131c05 service placement] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:12:30.271111 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c535d1d5-b016-4d77-89e3-c833d2131c05 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:12:30.272844 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-c535d1d5-b016-4d77-89e3-c833d2131c05 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:12:30.284036 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c535d1d5-b016-4d77-89e3-c833d2131c05 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:12:30.285602 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c535d1d5-b016-4d77-89e3-c833d2131c05 service placement] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:12:30.295430 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1409/2794] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 10:12:30 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:12:38.744102 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d7b62b-5d7b-45c6-9b7f-4bac9e224675 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:12:38.745572 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-f3d7b62b-5d7b-45c6-9b7f-4bac9e224675 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:12:38.825420 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d7b62b-5d7b-45c6-9b7f-4bac9e224675 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:12:38.829697 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbca1566-72bb-407c-b433-a6543c9cb3bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e5260e798404e8bb12b25ca8ea97052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:12:38.830304 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbca1566-72bb-407c-b433-a6543c9cb3bb service neutron] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:12:38.830586 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbca1566-72bb-407c-b433-a6543c9cb3bb service neutron] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:12:38.830860 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbca1566-72bb-407c-b433-a6543c9cb3bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:12:38.832658 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-fbca1566-72bb-407c-b433-a6543c9cb3bb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:12:38.916765 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbca1566-72bb-407c-b433-a6543c9cb3bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:12:38.918109 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbca1566-72bb-407c-b433-a6543c9cb3bb service neutron] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:12:38.936775 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1386/2795] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 10:12:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:12:40.291217 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=58 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:12:42.290236 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c535d1d5-b016-4d77-89e3-c833d2131c05 service placement] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:12:42.291843 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-c535d1d5-b016-4d77-89e3-c833d2131c05 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:12:42.307561 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c535d1d5-b016-4d77-89e3-c833d2131c05 service placement] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:12:42.312164 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bbf096-744a-4e3b-84d8-633f3eb1e10f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80cada22039d4f11935412ee578acd05', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:12:42.313117 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bbf096-744a-4e3b-84d8-633f3eb1e10f service placement] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:12:42.313404 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bbf096-744a-4e3b-84d8-633f3eb1e10f service placement] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:12:42.313699 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bbf096-744a-4e3b-84d8-633f3eb1e10f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:12:42.315441 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-c9bbf096-744a-4e3b-84d8-633f3eb1e10f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:12:42.328480 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bbf096-744a-4e3b-84d8-633f3eb1e10f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:12:42.330079 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bbf096-744a-4e3b-84d8-633f3eb1e10f service placement] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:12:42.340685 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1410/2796] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 10:12:42 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:12:42.466956 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbca1566-72bb-407c-b433-a6543c9cb3bb service neutron] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:12:42.468454 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-fbca1566-72bb-407c-b433-a6543c9cb3bb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:12:42.483400 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbca1566-72bb-407c-b433-a6543c9cb3bb service neutron] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:12:42.487011 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5ee197-dcd6-40df-8c84-946df4bb3740 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:12:42.487722 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5ee197-dcd6-40df-8c84-946df4bb3740 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:12:42.488030 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5ee197-dcd6-40df-8c84-946df4bb3740 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:12:42.488300 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5ee197-dcd6-40df-8c84-946df4bb3740 admin admin] PATH_INFO: `/v3/users/def04335e57e48409a6943fa231d47c3` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:12:42.489401 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5ee197-dcd6-40df-8c84-946df4bb3740 admin admin] RBAC: Authorizing `identity:delete_user(user_id=def04335e57e48409a6943fa231d47c3)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:12:42.491417 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5ee197-dcd6-40df-8c84-946df4bb3740 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:12:42.709067 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-0a5ee197-dcd6-40df-8c84-946df4bb3740 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'def04335e57e48409a6943fa231d47c3'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:12:42.709712 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-0a5ee197-dcd6-40df-8c84-946df4bb3740 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'def04335e57e48409a6943fa231d47c3'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:12:42.716838 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:12:42.722656 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-0a5ee197-dcd6-40df-8c84-946df4bb3740 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'def04335e57e48409a6943fa231d47c3'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:12:42.729349 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-0a5ee197-dcd6-40df-8c84-946df4bb3740 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'def04335e57e48409a6943fa231d47c3'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:12:42.815027 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-0a5ee197-dcd6-40df-8c84-946df4bb3740 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'def04335e57e48409a6943fa231d47c3'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:12:42.822159 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1387/2797] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 24 10:12:42 2022] DELETE /identity/v3/users/def04335e57e48409a6943fa231d47c3 => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:12:43.114136 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bbf096-744a-4e3b-84d8-633f3eb1e10f service placement] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:12:43.125408 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bbf096-744a-4e3b-84d8-633f3eb1e10f service placement] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:12:43.181531 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5660874b-28a2-4ab8-81b6-053c23d0a6e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:12:43.182160 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5660874b-28a2-4ab8-81b6-053c23d0a6e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:12:43.182438 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5660874b-28a2-4ab8-81b6-053c23d0a6e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:12:43.182738 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5660874b-28a2-4ab8-81b6-053c23d0a6e6 admin admin] PATH_INFO: `/v3/projects/a99aef25597a4ab480625b53c48cfe80` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:12:43.183903 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5660874b-28a2-4ab8-81b6-053c23d0a6e6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a99aef25597a4ab480625b53c48cfe80)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:12:43.185860 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5660874b-28a2-4ab8-81b6-053c23d0a6e6 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:12:43.314975 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-5660874b-28a2-4ab8-81b6-053c23d0a6e6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a99aef25597a4ab480625b53c48cfe80', 'request_id': 'req-5660874b-28a2-4ab8-81b6-053c23d0a6e6'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:12:43.321135 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-5660874b-28a2-4ab8-81b6-053c23d0a6e6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a99aef25597a4ab480625b53c48cfe80', 'request_id': 'req-5660874b-28a2-4ab8-81b6-053c23d0a6e6'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:12:43.328400 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-5660874b-28a2-4ab8-81b6-053c23d0a6e6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a99aef25597a4ab480625b53c48cfe80', 'request_id': 'req-5660874b-28a2-4ab8-81b6-053c23d0a6e6'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:12:43.419904 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-5660874b-28a2-4ab8-81b6-053c23d0a6e6 admin admin] The token cache is being invalidate because project a99aef25597a4ab480625b53c48cfe80 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81602) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:12:43.420133 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-5660874b-28a2-4ab8-81b6-053c23d0a6e6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:12:43.420779 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-5660874b-28a2-4ab8-81b6-053c23d0a6e6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:12:43.422305 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1411/2798] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 24 10:12:43 2022] DELETE /identity/v3/projects/a99aef25597a4ab480625b53c48cfe80 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:12:43.428076 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3f31af-4953-4d58-b473-475e73ef2843 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:12:43.428368 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3f31af-4953-4d58-b473-475e73ef2843 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:12:43.428647 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3f31af-4953-4d58-b473-475e73ef2843 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:12:43.448067 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-3f3f31af-4953-4d58-b473-475e73ef2843 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:12:43.449418 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-3f3f31af-4953-4d58-b473-475e73ef2843 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:12:43.507023 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1388/2799] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 10:12:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:12:43.515093 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78248297-e5f5-43ac-9e03-695ae3e8a3ae None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:12:43.515379 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78248297-e5f5-43ac-9e03-695ae3e8a3ae None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:12:43.515704 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78248297-e5f5-43ac-9e03-695ae3e8a3ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:12:43.535358 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-78248297-e5f5-43ac-9e03-695ae3e8a3ae None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:12:43.536601 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-78248297-e5f5-43ac-9e03-695ae3e8a3ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:12:43.544384 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1412/2800] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 10:12:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:12:43.548396 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3f31af-4953-4d58-b473-475e73ef2843 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:12:43.557759 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3f31af-4953-4d58-b473-475e73ef2843 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:12:43.562345 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8e9072-6578-4eeb-af6e-020f107d7dd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:12:43.562944 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8e9072-6578-4eeb-af6e-020f107d7dd7 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:12:43.563224 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8e9072-6578-4eeb-af6e-020f107d7dd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:12:43.563499 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8e9072-6578-4eeb-af6e-020f107d7dd7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:12:43.564177 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8e9072-6578-4eeb-af6e-020f107d7dd7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:12:43.564484 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8e9072-6578-4eeb-af6e-020f107d7dd7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:12:43.566064 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8e9072-6578-4eeb-af6e-020f107d7dd7 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:12:43.577783 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1389/2801] 10.222.0.16 () {58 vars in 1153 bytes} [Wed Aug 24 10:12:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:12:43.581765 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78248297-e5f5-43ac-9e03-695ae3e8a3ae None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:12:43.590673 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78248297-e5f5-43ac-9e03-695ae3e8a3ae None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:12:43.596183 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec3b154-b197-4216-84b3-9d504981a66e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:12:43.596766 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec3b154-b197-4216-84b3-9d504981a66e admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:12:43.597041 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec3b154-b197-4216-84b3-9d504981a66e admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:12:43.597310 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec3b154-b197-4216-84b3-9d504981a66e admin admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:12:43.597927 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec3b154-b197-4216-84b3-9d504981a66e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:12:43.599825 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec3b154-b197-4216-84b3-9d504981a66e admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:12:43.690944 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1413/2802] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 24 10:12:43 2022] POST /identity/v3/projects => generated 405 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:12:43.695349 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8e9072-6578-4eeb-af6e-020f107d7dd7 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:12:43.706238 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8e9072-6578-4eeb-af6e-020f107d7dd7 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:12:43.765686 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c81cdd2-c2ae-4b10-9d32-f6f57b43a7c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:12:43.766309 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c81cdd2-c2ae-4b10-9d32-f6f57b43a7c8 admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:12:43.766589 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c81cdd2-c2ae-4b10-9d32-f6f57b43a7c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:12:43.766890 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c81cdd2-c2ae-4b10-9d32-f6f57b43a7c8 admin admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:12:43.767545 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c81cdd2-c2ae-4b10-9d32-f6f57b43a7c8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:12:43.769491 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c81cdd2-c2ae-4b10-9d32-f6f57b43a7c8 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:12:43.896140 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1390/2803] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 24 10:12:43 2022] POST /identity/v3/users => generated 303 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:12:43.900195 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec3b154-b197-4216-84b3-9d504981a66e admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:12:43.910335 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec3b154-b197-4216-84b3-9d504981a66e admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:12:43.914825 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5faf9d-7537-45cd-b2a7-1b824e705fa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:12:43.915417 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5faf9d-7537-45cd-b2a7-1b824e705fa2 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:12:43.915738 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5faf9d-7537-45cd-b2a7-1b824e705fa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:12:43.916014 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5faf9d-7537-45cd-b2a7-1b824e705fa2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:12:43.916650 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5faf9d-7537-45cd-b2a7-1b824e705fa2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:12:43.918231 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5faf9d-7537-45cd-b2a7-1b824e705fa2 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:12:43.928387 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1414/2804] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 24 10:12:43 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:12:43.932191 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c81cdd2-c2ae-4b10-9d32-f6f57b43a7c8 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:12:43.942517 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c81cdd2-c2ae-4b10-9d32-f6f57b43a7c8 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:12:43.947044 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c09b65c-7811-441a-9a80-91bffe2e38f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:12:43.947942 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c09b65c-7811-441a-9a80-91bffe2e38f2 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:12:43.947942 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c09b65c-7811-441a-9a80-91bffe2e38f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:12:43.948212 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c09b65c-7811-441a-9a80-91bffe2e38f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:12:43.948815 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c09b65c-7811-441a-9a80-91bffe2e38f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:12:43.950789 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c09b65c-7811-441a-9a80-91bffe2e38f2 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:12:43.961575 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1391/2805] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 24 10:12:43 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 10:12:43.965621 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5faf9d-7537-45cd-b2a7-1b824e705fa2 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:12:43.976057 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5faf9d-7537-45cd-b2a7-1b824e705fa2 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:12:43.980576 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1881e1-9f5b-4081-9e0e-98a1233c09fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:12:43.981211 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1881e1-9f5b-4081-9e0e-98a1233c09fc admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:12:43.981488 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1881e1-9f5b-4081-9e0e-98a1233c09fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:12:43.981760 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1881e1-9f5b-4081-9e0e-98a1233c09fc admin admin] PATH_INFO: `/v3/projects/053848ad21194c6ea29b3fb19b8c5821/users/3dec6845b46545bfbd8a39a1f8c711fa/roles/136cdf78e00a426997dd4e37faff489b` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:12:43.996724 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1881e1-9f5b-4081-9e0e-98a1233c09fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=053848ad21194c6ea29b3fb19b8c5821, user_id=3dec6845b46545bfbd8a39a1f8c711fa, role_id=136cdf78e00a426997dd4e37faff489b)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:12:44.000495 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1881e1-9f5b-4081-9e0e-98a1233c09fc admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:12:44.077909 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1415/2806] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 24 10:12:43 2022] PUT /identity/v3/projects/053848ad21194c6ea29b3fb19b8c5821/users/3dec6845b46545bfbd8a39a1f8c711fa/roles/136cdf78e00a426997dd4e37faff489b => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 10:12:44.082872 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03897853-9b5a-43ae-a170-4559734a4733 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:12:44.083158 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03897853-9b5a-43ae-a170-4559734a4733 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:12:44.083430 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03897853-9b5a-43ae-a170-4559734a4733 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:12:44.102619 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-03897853-9b5a-43ae-a170-4559734a4733 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:12:44.103859 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-03897853-9b5a-43ae-a170-4559734a4733 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:12:44.137749 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:12:44.162253 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1392/2807] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 10:12:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:12:44.172439 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1881e1-9f5b-4081-9e0e-98a1233c09fc admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:12:44.174169 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-9c1881e1-9f5b-4081-9e0e-98a1233c09fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:12:44.189205 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:12:44.231368 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1881e1-9f5b-4081-9e0e-98a1233c09fc admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:12:44.235937 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd9e2b3-30f8-422b-bc71-17232ab7f5e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e5260e798404e8bb12b25ca8ea97052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:12:44.236530 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd9e2b3-30f8-422b-bc71-17232ab7f5e3 service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:12:44.236805 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd9e2b3-30f8-422b-bc71-17232ab7f5e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:12:44.237071 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd9e2b3-30f8-422b-bc71-17232ab7f5e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:12:44.245666 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd9e2b3-30f8-422b-bc71-17232ab7f5e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:12:44.247071 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd9e2b3-30f8-422b-bc71-17232ab7f5e3 service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:12:44.256747 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1416/2808] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 10:12:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:12:45.805428 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae36f95-56dc-4b2d-9368-d22a23b5bd8e None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:12:45.805898 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae36f95-56dc-4b2d-9368-d22a23b5bd8e None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:12:45.806008 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae36f95-56dc-4b2d-9368-d22a23b5bd8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:12:45.834122 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-dae36f95-56dc-4b2d-9368-d22a23b5bd8e None None] MFA Rules not processed for user `3dec6845b46545bfbd8a39a1f8c711fa`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:12:45.835347 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-dae36f95-56dc-4b2d-9368-d22a23b5bd8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:12:45.909081 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1393/2809] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 10:12:45 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:12:45.913635 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd9e2b3-30f8-422b-bc71-17232ab7f5e3 service neutron] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:12:45.923794 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd9e2b3-30f8-422b-bc71-17232ab7f5e3 service neutron] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:12:45.928380 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755579ae-3ab9-4a38-a0f4-09cdab02afb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:12:45.928969 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755579ae-3ab9-4a38-a0f4-09cdab02afb1 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:12:45.929253 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755579ae-3ab9-4a38-a0f4-09cdab02afb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:12:45.929535 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755579ae-3ab9-4a38-a0f4-09cdab02afb1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:12:45.930165 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755579ae-3ab9-4a38-a0f4-09cdab02afb1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:12:45.932098 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755579ae-3ab9-4a38-a0f4-09cdab02afb1 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:12:46.005440 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1417/2810] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 24 10:12:45 2022] POST /identity/v3/projects => generated 403 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:12:46.009822 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae36f95-56dc-4b2d-9368-d22a23b5bd8e None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:12:46.020667 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae36f95-56dc-4b2d-9368-d22a23b5bd8e None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:12:46.077647 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c7761b-3657-4507-b000-a2ff3add16a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:12:46.078169 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c7761b-3657-4507-b000-a2ff3add16a2 admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:12:46.078445 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c7761b-3657-4507-b000-a2ff3add16a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:12:46.078713 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c7761b-3657-4507-b000-a2ff3add16a2 admin admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:12:46.079333 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c7761b-3657-4507-b000-a2ff3add16a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:12:46.081269 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c7761b-3657-4507-b000-a2ff3add16a2 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:12:46.172055 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1394/2811] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 24 10:12:46 2022] POST /identity/v3/users => generated 308 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:12:46.176105 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755579ae-3ab9-4a38-a0f4-09cdab02afb1 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:12:46.186633 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755579ae-3ab9-4a38-a0f4-09cdab02afb1 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:12:46.191195 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4724187d-8496-4b91-bea2-a9c3e4f63d87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:12:46.191824 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4724187d-8496-4b91-bea2-a9c3e4f63d87 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:12:46.192110 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4724187d-8496-4b91-bea2-a9c3e4f63d87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:12:46.192387 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4724187d-8496-4b91-bea2-a9c3e4f63d87 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:12:46.193030 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4724187d-8496-4b91-bea2-a9c3e4f63d87 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:12:46.194617 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4724187d-8496-4b91-bea2-a9c3e4f63d87 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:12:46.204858 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1418/2812] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 24 10:12:46 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:12:46.208739 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c7761b-3657-4507-b000-a2ff3add16a2 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:12:46.219038 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c7761b-3657-4507-b000-a2ff3add16a2 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:12:46.224109 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b84adb-aa74-4b21-a28d-2a46f1102f5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:12:46.224757 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b84adb-aa74-4b21-a28d-2a46f1102f5c admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:12:46.225039 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b84adb-aa74-4b21-a28d-2a46f1102f5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:12:46.225311 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b84adb-aa74-4b21-a28d-2a46f1102f5c admin admin] PATH_INFO: `/v3/projects/29f4945330424ce2b455ccf7e8b17510/users/7167c05b3d334cbd9816db0a55738d99/roles/15f634d4a2d247ddab90c458e5e1c141` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:12:46.240767 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b84adb-aa74-4b21-a28d-2a46f1102f5c admin admin] RBAC: Authorizing `identity:create_grant(project_id=29f4945330424ce2b455ccf7e8b17510, user_id=7167c05b3d334cbd9816db0a55738d99, role_id=15f634d4a2d247ddab90c458e5e1c141)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:12:46.243407 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b84adb-aa74-4b21-a28d-2a46f1102f5c admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:12:46.317784 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1395/2813] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 24 10:12:46 2022] PUT /identity/v3/projects/29f4945330424ce2b455ccf7e8b17510/users/7167c05b3d334cbd9816db0a55738d99/roles/15f634d4a2d247ddab90c458e5e1c141 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 10:12:46.321680 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4724187d-8496-4b91-bea2-a9c3e4f63d87 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:12:46.331884 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4724187d-8496-4b91-bea2-a9c3e4f63d87 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:12:46.395330 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5d607b-d9f3-4e41-bf6d-25461203858b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:12:46.395961 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5d607b-d9f3-4e41-bf6d-25461203858b admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:12:46.396237 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5d607b-d9f3-4e41-bf6d-25461203858b admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:12:46.396506 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5d607b-d9f3-4e41-bf6d-25461203858b admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:12:46.397167 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5d607b-d9f3-4e41-bf6d-25461203858b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:12:46.398752 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5d607b-d9f3-4e41-bf6d-25461203858b admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:12:46.408804 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1419/2814] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 24 10:12:46 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:12:46.412842 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b84adb-aa74-4b21-a28d-2a46f1102f5c admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:12:46.423093 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b84adb-aa74-4b21-a28d-2a46f1102f5c admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:12:46.427736 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649f7f0b-a1ed-454c-a69b-9f94e20531c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:12:46.428386 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649f7f0b-a1ed-454c-a69b-9f94e20531c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:12:46.428668 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649f7f0b-a1ed-454c-a69b-9f94e20531c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:12:46.428940 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649f7f0b-a1ed-454c-a69b-9f94e20531c2 admin admin] PATH_INFO: `/v3/projects/29f4945330424ce2b455ccf7e8b17510/users/7167c05b3d334cbd9816db0a55738d99/roles/15f634d4a2d247ddab90c458e5e1c141` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:12:46.430870 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649f7f0b-a1ed-454c-a69b-9f94e20531c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=29f4945330424ce2b455ccf7e8b17510, user_id=7167c05b3d334cbd9816db0a55738d99, role_id=15f634d4a2d247ddab90c458e5e1c141)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:12:46.433526 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649f7f0b-a1ed-454c-a69b-9f94e20531c2 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:12:46.443697 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1396/2815] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 24 10:12:46 2022] PUT /identity/v3/projects/29f4945330424ce2b455ccf7e8b17510/users/7167c05b3d334cbd9816db0a55738d99/roles/15f634d4a2d247ddab90c458e5e1c141 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:12:47.718269 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f4657c-c3ec-4871-9977-78a9e37ed1fd None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:12:47.719239 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f4657c-c3ec-4871-9977-78a9e37ed1fd None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:12:47.719985 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f4657c-c3ec-4871-9977-78a9e37ed1fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:12:47.750277 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-85f4657c-c3ec-4871-9977-78a9e37ed1fd None None] MFA Rules not processed for user `7167c05b3d334cbd9816db0a55738d99`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:12:47.751744 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-85f4657c-c3ec-4871-9977-78a9e37ed1fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:12:47.823670 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1420/2816] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 10:12:47 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:12:49.152925 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649f7f0b-a1ed-454c-a69b-9f94e20531c2 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:12:49.178354 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:12:49.215385 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649f7f0b-a1ed-454c-a69b-9f94e20531c2 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:12:49.220781 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c64e2fb-48f5-4bbd-9dee-ed6bea50b8c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:12:49.229712 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c64e2fb-48f5-4bbd-9dee-ed6bea50b8c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:12:49.229712 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c64e2fb-48f5-4bbd-9dee-ed6bea50b8c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:12:49.229712 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c64e2fb-48f5-4bbd-9dee-ed6bea50b8c6 admin admin] PATH_INFO: `/v3/users/3dec6845b46545bfbd8a39a1f8c711fa` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:12:49.229712 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c64e2fb-48f5-4bbd-9dee-ed6bea50b8c6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3dec6845b46545bfbd8a39a1f8c711fa)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:12:49.229712 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c64e2fb-48f5-4bbd-9dee-ed6bea50b8c6 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:12:49.425639 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-2c64e2fb-48f5-4bbd-9dee-ed6bea50b8c6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3dec6845b46545bfbd8a39a1f8c711fa'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:12:49.426219 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-2c64e2fb-48f5-4bbd-9dee-ed6bea50b8c6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3dec6845b46545bfbd8a39a1f8c711fa'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:12:49.438836 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-2c64e2fb-48f5-4bbd-9dee-ed6bea50b8c6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3dec6845b46545bfbd8a39a1f8c711fa'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:12:49.445663 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-2c64e2fb-48f5-4bbd-9dee-ed6bea50b8c6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3dec6845b46545bfbd8a39a1f8c711fa'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:12:49.551846 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-2c64e2fb-48f5-4bbd-9dee-ed6bea50b8c6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3dec6845b46545bfbd8a39a1f8c711fa'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:12:49.557846 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1397/2817] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 24 10:12:49 2022] DELETE /identity/v3/users/3dec6845b46545bfbd8a39a1f8c711fa => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:12:49.792923 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f4657c-c3ec-4871-9977-78a9e37ed1fd None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:12:49.805512 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:12:49.806820 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f4657c-c3ec-4871-9977-78a9e37ed1fd None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:12:49.864177 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03b3fb7-2bdd-48bf-afdb-bf56ef4ce69f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:12:49.864806 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03b3fb7-2bdd-48bf-afdb-bf56ef4ce69f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:12:49.865113 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03b3fb7-2bdd-48bf-afdb-bf56ef4ce69f admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:12:49.865388 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03b3fb7-2bdd-48bf-afdb-bf56ef4ce69f admin admin] PATH_INFO: `/v3/projects/053848ad21194c6ea29b3fb19b8c5821` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:12:49.866507 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03b3fb7-2bdd-48bf-afdb-bf56ef4ce69f admin admin] RBAC: Authorizing `identity:delete_project(project_id=053848ad21194c6ea29b3fb19b8c5821)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:12:49.868444 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03b3fb7-2bdd-48bf-afdb-bf56ef4ce69f admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:12:49.994768 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-b03b3fb7-2bdd-48bf-afdb-bf56ef4ce69f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '053848ad21194c6ea29b3fb19b8c5821', 'request_id': 'req-b03b3fb7-2bdd-48bf-afdb-bf56ef4ce69f'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:12:50.000967 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-b03b3fb7-2bdd-48bf-afdb-bf56ef4ce69f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '053848ad21194c6ea29b3fb19b8c5821', 'request_id': 'req-b03b3fb7-2bdd-48bf-afdb-bf56ef4ce69f'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:12:50.006701 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-b03b3fb7-2bdd-48bf-afdb-bf56ef4ce69f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '053848ad21194c6ea29b3fb19b8c5821', 'request_id': 'req-b03b3fb7-2bdd-48bf-afdb-bf56ef4ce69f'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:12:50.112061 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-b03b3fb7-2bdd-48bf-afdb-bf56ef4ce69f admin admin] The token cache is being invalidate because project 053848ad21194c6ea29b3fb19b8c5821 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81602) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:12:50.112283 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-b03b3fb7-2bdd-48bf-afdb-bf56ef4ce69f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:12:50.112902 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-b03b3fb7-2bdd-48bf-afdb-bf56ef4ce69f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:12:50.114405 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1421/2818] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 24 10:12:49 2022] DELETE /identity/v3/projects/053848ad21194c6ea29b3fb19b8c5821 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:12:50.118338 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c64e2fb-48f5-4bbd-9dee-ed6bea50b8c6 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:12:50.119819 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-2c64e2fb-48f5-4bbd-9dee-ed6bea50b8c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:12:50.132715 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c64e2fb-48f5-4bbd-9dee-ed6bea50b8c6 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:12:50.188331 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74784ba8-fd1a-433c-a5e0-b5471cbfe0ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:12:50.189009 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74784ba8-fd1a-433c-a5e0-b5471cbfe0ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:12:50.189221 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74784ba8-fd1a-433c-a5e0-b5471cbfe0ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:12:50.189488 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74784ba8-fd1a-433c-a5e0-b5471cbfe0ad admin admin] PATH_INFO: `/v3/users/7167c05b3d334cbd9816db0a55738d99` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:12:50.190598 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74784ba8-fd1a-433c-a5e0-b5471cbfe0ad admin admin] RBAC: Authorizing `identity:delete_user(user_id=7167c05b3d334cbd9816db0a55738d99)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:12:50.192524 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74784ba8-fd1a-433c-a5e0-b5471cbfe0ad admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:12:50.405430 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:12:50.405430 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-74784ba8-fd1a-433c-a5e0-b5471cbfe0ad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7167c05b3d334cbd9816db0a55738d99'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:12:50.406000 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-74784ba8-fd1a-433c-a5e0-b5471cbfe0ad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7167c05b3d334cbd9816db0a55738d99'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:12:50.419221 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-74784ba8-fd1a-433c-a5e0-b5471cbfe0ad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7167c05b3d334cbd9816db0a55738d99'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:12:50.426022 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-74784ba8-fd1a-433c-a5e0-b5471cbfe0ad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7167c05b3d334cbd9816db0a55738d99'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:12:50.528767 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-74784ba8-fd1a-433c-a5e0-b5471cbfe0ad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7167c05b3d334cbd9816db0a55738d99'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:12:50.535149 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1398/2819] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 24 10:12:50 2022] DELETE /identity/v3/users/7167c05b3d334cbd9816db0a55738d99 => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:12:50.799162 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03b3fb7-2bdd-48bf-afdb-bf56ef4ce69f admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:12:50.800702 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-b03b3fb7-2bdd-48bf-afdb-bf56ef4ce69f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:12:50.814100 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03b3fb7-2bdd-48bf-afdb-bf56ef4ce69f admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:12:50.873942 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7534a8b0-7098-4807-9f8f-a7ba02a5891d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:12:50.875071 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7534a8b0-7098-4807-9f8f-a7ba02a5891d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:12:50.875354 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7534a8b0-7098-4807-9f8f-a7ba02a5891d admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:12:50.875664 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7534a8b0-7098-4807-9f8f-a7ba02a5891d admin admin] PATH_INFO: `/v3/projects/29f4945330424ce2b455ccf7e8b17510` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:12:50.876785 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7534a8b0-7098-4807-9f8f-a7ba02a5891d admin admin] RBAC: Authorizing `identity:delete_project(project_id=29f4945330424ce2b455ccf7e8b17510)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:12:50.878741 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7534a8b0-7098-4807-9f8f-a7ba02a5891d admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:12:51.039748 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:12:51.048666 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-7534a8b0-7098-4807-9f8f-a7ba02a5891d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '29f4945330424ce2b455ccf7e8b17510', 'request_id': 'req-7534a8b0-7098-4807-9f8f-a7ba02a5891d'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:12:51.054743 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-7534a8b0-7098-4807-9f8f-a7ba02a5891d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '29f4945330424ce2b455ccf7e8b17510', 'request_id': 'req-7534a8b0-7098-4807-9f8f-a7ba02a5891d'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:12:51.060071 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-7534a8b0-7098-4807-9f8f-a7ba02a5891d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '29f4945330424ce2b455ccf7e8b17510', 'request_id': 'req-7534a8b0-7098-4807-9f8f-a7ba02a5891d'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:12:51.134282 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-7534a8b0-7098-4807-9f8f-a7ba02a5891d admin admin] The token cache is being invalidate because project 29f4945330424ce2b455ccf7e8b17510 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81602) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:12:51.134572 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-7534a8b0-7098-4807-9f8f-a7ba02a5891d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:12:51.135190 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-7534a8b0-7098-4807-9f8f-a7ba02a5891d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:12:51.136842 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1422/2820] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 24 10:12:50 2022] DELETE /identity/v3/projects/29f4945330424ce2b455ccf7e8b17510 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:12:51.141733 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af1639b-cb11-47b4-a7d1-ce196106bb15 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:12:51.142019 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af1639b-cb11-47b4-a7d1-ce196106bb15 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:12:51.142295 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af1639b-cb11-47b4-a7d1-ce196106bb15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:12:51.161691 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-8af1639b-cb11-47b4-a7d1-ce196106bb15 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:12:51.171777 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-8af1639b-cb11-47b4-a7d1-ce196106bb15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:12:51.222800 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1399/2821] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 10:12:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:12:51.230687 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbb048d-3518-4d22-8f55-e48ddac6cfe5 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:12:51.230977 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbb048d-3518-4d22-8f55-e48ddac6cfe5 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:12:51.231251 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbb048d-3518-4d22-8f55-e48ddac6cfe5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:12:51.251205 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-ffbb048d-3518-4d22-8f55-e48ddac6cfe5 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:12:51.252429 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-ffbb048d-3518-4d22-8f55-e48ddac6cfe5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:12:51.260271 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1423/2822] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 10:12:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:12:51.264472 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af1639b-cb11-47b4-a7d1-ce196106bb15 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:12:51.273871 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af1639b-cb11-47b4-a7d1-ce196106bb15 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:12:51.278413 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f60e7fe-fe0d-4366-bc40-02d319662063 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:12:51.278999 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f60e7fe-fe0d-4366-bc40-02d319662063 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:12:51.279390 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f60e7fe-fe0d-4366-bc40-02d319662063 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:12:51.279693 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f60e7fe-fe0d-4366-bc40-02d319662063 admin admin] PATH_INFO: `/v3/domains` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:12:51.280837 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f60e7fe-fe0d-4366-bc40-02d319662063 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:12:51.281131 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f60e7fe-fe0d-4366-bc40-02d319662063 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:12:51.282706 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f60e7fe-fe0d-4366-bc40-02d319662063 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:12:51.293897 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1400/2823] 10.222.0.16 () {58 vars in 1153 bytes} [Wed Aug 24 10:12:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:12:51.298720 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbb048d-3518-4d22-8f55-e48ddac6cfe5 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:12:51.308023 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbb048d-3518-4d22-8f55-e48ddac6cfe5 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:12:51.312692 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010ff1e7-7f4e-40f7-9c9c-21c4e776fee4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:12:51.313305 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010ff1e7-7f4e-40f7-9c9c-21c4e776fee4 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:12:51.313605 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010ff1e7-7f4e-40f7-9c9c-21c4e776fee4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:12:51.313877 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010ff1e7-7f4e-40f7-9c9c-21c4e776fee4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:12:51.314507 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010ff1e7-7f4e-40f7-9c9c-21c4e776fee4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:12:51.316462 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010ff1e7-7f4e-40f7-9c9c-21c4e776fee4 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:12:51.439017 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1424/2824] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 24 10:12:51 2022] POST /identity/v3/projects => generated 387 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:12:51.443207 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f60e7fe-fe0d-4366-bc40-02d319662063 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:12:51.453267 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f60e7fe-fe0d-4366-bc40-02d319662063 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:12:51.508514 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554fde33-f0e5-4b3c-8e75-cedcc776cf90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:12:51.509119 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554fde33-f0e5-4b3c-8e75-cedcc776cf90 admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:12:51.509394 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554fde33-f0e5-4b3c-8e75-cedcc776cf90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:12:51.509671 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554fde33-f0e5-4b3c-8e75-cedcc776cf90 admin admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:12:51.510293 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554fde33-f0e5-4b3c-8e75-cedcc776cf90 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:12:51.512284 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554fde33-f0e5-4b3c-8e75-cedcc776cf90 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:12:51.608676 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1401/2825] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 24 10:12:51 2022] POST /identity/v3/users => generated 294 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:12:51.612638 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010ff1e7-7f4e-40f7-9c9c-21c4e776fee4 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:12:51.623113 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010ff1e7-7f4e-40f7-9c9c-21c4e776fee4 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:12:51.627850 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30345ce-950e-4725-abf0-e6aa45f5898f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:12:51.628452 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30345ce-950e-4725-abf0-e6aa45f5898f admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:12:51.628732 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30345ce-950e-4725-abf0-e6aa45f5898f admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:12:51.629005 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30345ce-950e-4725-abf0-e6aa45f5898f admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:12:51.629646 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30345ce-950e-4725-abf0-e6aa45f5898f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:12:51.631230 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30345ce-950e-4725-abf0-e6aa45f5898f admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:12:51.641439 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1425/2826] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 24 10:12:51 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:12:51.645857 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554fde33-f0e5-4b3c-8e75-cedcc776cf90 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:12:51.656548 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554fde33-f0e5-4b3c-8e75-cedcc776cf90 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:12:51.661755 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446362c0-a9f0-4371-b06d-e2038a59b7da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:12:51.662355 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446362c0-a9f0-4371-b06d-e2038a59b7da admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:12:51.662630 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446362c0-a9f0-4371-b06d-e2038a59b7da admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:12:51.662899 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446362c0-a9f0-4371-b06d-e2038a59b7da admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:12:51.663532 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446362c0-a9f0-4371-b06d-e2038a59b7da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:12:51.665612 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446362c0-a9f0-4371-b06d-e2038a59b7da admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:12:51.676988 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1402/2827] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 24 10:12:51 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:12:51.681089 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30345ce-950e-4725-abf0-e6aa45f5898f admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:12:51.690958 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30345ce-950e-4725-abf0-e6aa45f5898f admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:12:51.695567 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d568c47-b1a8-42de-9d40-c22660550e3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:12:51.696230 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d568c47-b1a8-42de-9d40-c22660550e3e admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:12:51.696510 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d568c47-b1a8-42de-9d40-c22660550e3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:12:51.696782 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d568c47-b1a8-42de-9d40-c22660550e3e admin admin] PATH_INFO: `/v3/projects/e12d580f94f84cbfabdf2d933408f39c/users/2631384b65584ab397530e763c917224/roles/136cdf78e00a426997dd4e37faff489b` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:12:51.711297 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d568c47-b1a8-42de-9d40-c22660550e3e admin admin] RBAC: Authorizing `identity:create_grant(project_id=e12d580f94f84cbfabdf2d933408f39c, user_id=2631384b65584ab397530e763c917224, role_id=136cdf78e00a426997dd4e37faff489b)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:12:51.713961 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d568c47-b1a8-42de-9d40-c22660550e3e admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:12:51.822803 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1426/2828] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 24 10:12:51 2022] PUT /identity/v3/projects/e12d580f94f84cbfabdf2d933408f39c/users/2631384b65584ab397530e763c917224/roles/136cdf78e00a426997dd4e37faff489b => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:12:51.834728 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f0f461-7918-42ee-b8ac-dddaddd8ea25 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:12:51.835012 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f0f461-7918-42ee-b8ac-dddaddd8ea25 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:12:51.835282 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f0f461-7918-42ee-b8ac-dddaddd8ea25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:12:51.866419 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-20f0f461-7918-42ee-b8ac-dddaddd8ea25 None None] MFA Rules not processed for user `2631384b65584ab397530e763c917224`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:12:51.866419 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-20f0f461-7918-42ee-b8ac-dddaddd8ea25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:12:51.877072 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:12:51.929541 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1403/2829] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 10:12:51 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:12:51.934119 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d568c47-b1a8-42de-9d40-c22660550e3e admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:12:51.998194 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d568c47-b1a8-42de-9d40-c22660550e3e admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:12:52.003463 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9528c14-df48-4d23-b5f6-a3ab66723873 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:12:52.004138 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9528c14-df48-4d23-b5f6-a3ab66723873 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:12:52.004383 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9528c14-df48-4d23-b5f6-a3ab66723873 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:12:52.004657 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9528c14-df48-4d23-b5f6-a3ab66723873 admin admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:12:52.005291 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9528c14-df48-4d23-b5f6-a3ab66723873 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:12:52.007174 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9528c14-df48-4d23-b5f6-a3ab66723873 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:12:52.107649 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1427/2830] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 24 10:12:51 2022] POST /identity/v3/projects => generated 387 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:12:52.111872 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f0f461-7918-42ee-b8ac-dddaddd8ea25 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:12:52.122406 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f0f461-7918-42ee-b8ac-dddaddd8ea25 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:12:52.177430 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304dcb8b-1c23-4383-a77a-84f05ed7a239 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:12:52.178027 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304dcb8b-1c23-4383-a77a-84f05ed7a239 admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:12:52.178305 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304dcb8b-1c23-4383-a77a-84f05ed7a239 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:12:52.178574 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304dcb8b-1c23-4383-a77a-84f05ed7a239 admin admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:12:52.179190 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304dcb8b-1c23-4383-a77a-84f05ed7a239 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:12:52.181117 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304dcb8b-1c23-4383-a77a-84f05ed7a239 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:12:52.262765 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1404/2831] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 24 10:12:52 2022] POST /identity/v3/users => generated 300 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 10:12:52.267305 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9528c14-df48-4d23-b5f6-a3ab66723873 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:12:52.278274 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9528c14-df48-4d23-b5f6-a3ab66723873 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:12:52.283363 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e79a3c-133a-4f22-a2aa-9b6b58b158b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:12:52.284001 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e79a3c-133a-4f22-a2aa-9b6b58b158b2 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:12:52.284281 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e79a3c-133a-4f22-a2aa-9b6b58b158b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:12:52.284549 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e79a3c-133a-4f22-a2aa-9b6b58b158b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:12:52.285185 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e79a3c-133a-4f22-a2aa-9b6b58b158b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:12:52.286758 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e79a3c-133a-4f22-a2aa-9b6b58b158b2 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:12:52.298188 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1428/2832] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 24 10:12:52 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:12:52.302646 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304dcb8b-1c23-4383-a77a-84f05ed7a239 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:12:52.313685 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304dcb8b-1c23-4383-a77a-84f05ed7a239 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:12:52.318322 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb56eb4-e642-4b58-b8d8-6f670226a87f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:12:52.319010 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb56eb4-e642-4b58-b8d8-6f670226a87f admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:12:52.319285 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb56eb4-e642-4b58-b8d8-6f670226a87f admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:12:52.319555 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb56eb4-e642-4b58-b8d8-6f670226a87f admin admin] PATH_INFO: `/v3/projects/350a545dafcf4716a7771d2638e39325/users/5782cf31281d4606983371a755c8f25f/roles/15f634d4a2d247ddab90c458e5e1c141` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:12:52.334999 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb56eb4-e642-4b58-b8d8-6f670226a87f admin admin] RBAC: Authorizing `identity:create_grant(project_id=350a545dafcf4716a7771d2638e39325, user_id=5782cf31281d4606983371a755c8f25f, role_id=15f634d4a2d247ddab90c458e5e1c141)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:12:52.338225 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb56eb4-e642-4b58-b8d8-6f670226a87f admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:12:52.411657 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1405/2833] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 24 10:12:52 2022] PUT /identity/v3/projects/350a545dafcf4716a7771d2638e39325/users/5782cf31281d4606983371a755c8f25f/roles/15f634d4a2d247ddab90c458e5e1c141 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 10:12:52.415778 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e79a3c-133a-4f22-a2aa-9b6b58b158b2 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:12:52.426198 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e79a3c-133a-4f22-a2aa-9b6b58b158b2 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:12:52.457945 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:12:52.481411 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d79912a-a9c2-4862-8785-d70c628976bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:12:52.482028 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d79912a-a9c2-4862-8785-d70c628976bc admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:12:52.482313 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d79912a-a9c2-4862-8785-d70c628976bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:12:52.482583 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d79912a-a9c2-4862-8785-d70c628976bc admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:12:52.483731 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d79912a-a9c2-4862-8785-d70c628976bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:12:52.485407 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d79912a-a9c2-4862-8785-d70c628976bc admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:12:52.495515 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1429/2834] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 24 10:12:52 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:12:52.500369 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb56eb4-e642-4b58-b8d8-6f670226a87f admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:12:52.510604 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb56eb4-e642-4b58-b8d8-6f670226a87f admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:12:52.515108 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c445cb16-b551-4ff7-8cef-cd7be6d979d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:12:52.515788 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c445cb16-b551-4ff7-8cef-cd7be6d979d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:12:52.516123 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c445cb16-b551-4ff7-8cef-cd7be6d979d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:12:52.516464 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c445cb16-b551-4ff7-8cef-cd7be6d979d4 admin admin] PATH_INFO: `/v3/projects/350a545dafcf4716a7771d2638e39325/users/5782cf31281d4606983371a755c8f25f/roles/15f634d4a2d247ddab90c458e5e1c141` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:12:52.518337 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c445cb16-b551-4ff7-8cef-cd7be6d979d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=350a545dafcf4716a7771d2638e39325, user_id=5782cf31281d4606983371a755c8f25f, role_id=15f634d4a2d247ddab90c458e5e1c141)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:12:52.521115 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c445cb16-b551-4ff7-8cef-cd7be6d979d4 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:12:52.531225 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1406/2835] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 24 10:12:52 2022] PUT /identity/v3/projects/350a545dafcf4716a7771d2638e39325/users/5782cf31281d4606983371a755c8f25f/roles/15f634d4a2d247ddab90c458e5e1c141 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:12:52.541250 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f05bad-ef57-4560-89ee-7016c02413ac None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:12:52.541534 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f05bad-ef57-4560-89ee-7016c02413ac None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:12:52.541801 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f05bad-ef57-4560-89ee-7016c02413ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:12:52.570001 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-72f05bad-ef57-4560-89ee-7016c02413ac None None] MFA Rules not processed for user `5782cf31281d4606983371a755c8f25f`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:12:52.571210 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-72f05bad-ef57-4560-89ee-7016c02413ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:12:52.637074 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1430/2836] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 10:12:52 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:12:52.648293 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c445cb16-b551-4ff7-8cef-cd7be6d979d4 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:12:52.649961 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-c445cb16-b551-4ff7-8cef-cd7be6d979d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:12:52.718574 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c445cb16-b551-4ff7-8cef-cd7be6d979d4 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:12:52.722442 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb6105f-40eb-40ff-b314-f9e8463302ba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c32b83577c46488906fe409a790b24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:12:52.723062 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb6105f-40eb-40ff-b314-f9e8463302ba service cinder] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:12:52.723341 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb6105f-40eb-40ff-b314-f9e8463302ba service cinder] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:12:52.723645 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb6105f-40eb-40ff-b314-f9e8463302ba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:12:52.732472 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb6105f-40eb-40ff-b314-f9e8463302ba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:12:52.734449 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb6105f-40eb-40ff-b314-f9e8463302ba service cinder] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:12:52.739335 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:12:52.744019 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1407/2837] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 10:12:52 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:12:52.771399 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f05bad-ef57-4560-89ee-7016c02413ac None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:12:52.832683 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f05bad-ef57-4560-89ee-7016c02413ac None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:12:52.837589 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9353e0b-f3d8-49f9-b319-9173d03f13d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:12:52.838227 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9353e0b-f3d8-49f9-b319-9173d03f13d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:12:52.838504 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9353e0b-f3d8-49f9-b319-9173d03f13d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:12:52.838777 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9353e0b-f3d8-49f9-b319-9173d03f13d1 admin admin] PATH_INFO: `/v3/users/2631384b65584ab397530e763c917224` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:12:52.839987 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9353e0b-f3d8-49f9-b319-9173d03f13d1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2631384b65584ab397530e763c917224)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:12:52.841891 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9353e0b-f3d8-49f9-b319-9173d03f13d1 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:12:53.062610 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-b9353e0b-f3d8-49f9-b319-9173d03f13d1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2631384b65584ab397530e763c917224'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:12:53.063336 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-b9353e0b-f3d8-49f9-b319-9173d03f13d1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2631384b65584ab397530e763c917224'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:12:53.157145 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-b9353e0b-f3d8-49f9-b319-9173d03f13d1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2631384b65584ab397530e763c917224'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:12:53.157757 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-b9353e0b-f3d8-49f9-b319-9173d03f13d1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2631384b65584ab397530e763c917224'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:12:53.162482 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:12:53.165328 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-b9353e0b-f3d8-49f9-b319-9173d03f13d1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2631384b65584ab397530e763c917224'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:12:53.182190 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1431/2838] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 24 10:12:52 2022] DELETE /identity/v3/users/2631384b65584ab397530e763c917224 => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:12:53.186672 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a6600d-34b1-43d8-9971-d239a5815c65 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:12:53.186975 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a6600d-34b1-43d8-9971-d239a5815c65 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:12:53.187260 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a6600d-34b1-43d8-9971-d239a5815c65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:12:53.207789 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-85a6600d-34b1-43d8-9971-d239a5815c65 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:12:53.209308 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-85a6600d-34b1-43d8-9971-d239a5815c65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:12:53.267797 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1408/2839] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 10:12:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:12:53.277470 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9353e0b-f3d8-49f9-b319-9173d03f13d1 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:12:53.280991 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-b9353e0b-f3d8-49f9-b319-9173d03f13d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:12:53.336673 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9353e0b-f3d8-49f9-b319-9173d03f13d1 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:12:53.340202 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd66d20e-e6c6-4db1-944e-689de820c6ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e5260e798404e8bb12b25ca8ea97052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:12:53.340807 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd66d20e-e6c6-4db1-944e-689de820c6ce service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:12:53.341086 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd66d20e-e6c6-4db1-944e-689de820c6ce service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:12:53.341365 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd66d20e-e6c6-4db1-944e-689de820c6ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:12:53.351099 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd66d20e-e6c6-4db1-944e-689de820c6ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:12:53.352563 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd66d20e-e6c6-4db1-944e-689de820c6ce service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:12:53.361732 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1432/2840] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 10:12:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:12:53.796379 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a6600d-34b1-43d8-9971-d239a5815c65 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:12:53.797827 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-85a6600d-34b1-43d8-9971-d239a5815c65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:12:53.812906 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a6600d-34b1-43d8-9971-d239a5815c65 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:12:53.818931 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933e2beb-cbbc-4251-b7db-d1afee15a2cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:12:53.819527 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933e2beb-cbbc-4251-b7db-d1afee15a2cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:12:53.819847 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933e2beb-cbbc-4251-b7db-d1afee15a2cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:12:53.820173 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933e2beb-cbbc-4251-b7db-d1afee15a2cc admin admin] PATH_INFO: `/v3/projects/e12d580f94f84cbfabdf2d933408f39c` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:12:53.821260 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933e2beb-cbbc-4251-b7db-d1afee15a2cc admin admin] RBAC: Authorizing `identity:delete_project(project_id=e12d580f94f84cbfabdf2d933408f39c)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:12:53.823116 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933e2beb-cbbc-4251-b7db-d1afee15a2cc admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:12:53.933379 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-933e2beb-cbbc-4251-b7db-d1afee15a2cc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e12d580f94f84cbfabdf2d933408f39c', 'request_id': 'req-933e2beb-cbbc-4251-b7db-d1afee15a2cc'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:12:53.940571 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-933e2beb-cbbc-4251-b7db-d1afee15a2cc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e12d580f94f84cbfabdf2d933408f39c', 'request_id': 'req-933e2beb-cbbc-4251-b7db-d1afee15a2cc'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:12:53.946169 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-933e2beb-cbbc-4251-b7db-d1afee15a2cc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e12d580f94f84cbfabdf2d933408f39c', 'request_id': 'req-933e2beb-cbbc-4251-b7db-d1afee15a2cc'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:12:54.025249 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-933e2beb-cbbc-4251-b7db-d1afee15a2cc admin admin] The token cache is being invalidate because project e12d580f94f84cbfabdf2d933408f39c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:12:54.025539 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-933e2beb-cbbc-4251-b7db-d1afee15a2cc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:12:54.026161 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-933e2beb-cbbc-4251-b7db-d1afee15a2cc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:12:54.028051 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1409/2841] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 24 10:12:53 2022] DELETE /identity/v3/projects/e12d580f94f84cbfabdf2d933408f39c => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:12:54.032046 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd66d20e-e6c6-4db1-944e-689de820c6ce service neutron] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:12:54.041580 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd66d20e-e6c6-4db1-944e-689de820c6ce service neutron] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:12:54.097661 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5faa72be-3ad8-476f-b481-9a1f98396d9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:12:54.098284 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faa72be-3ad8-476f-b481-9a1f98396d9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:12:54.098566 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faa72be-3ad8-476f-b481-9a1f98396d9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:12:54.098841 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faa72be-3ad8-476f-b481-9a1f98396d9a admin admin] PATH_INFO: `/v3/users/5782cf31281d4606983371a755c8f25f` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:12:54.099993 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5faa72be-3ad8-476f-b481-9a1f98396d9a admin admin] RBAC: Authorizing `identity:delete_user(user_id=5782cf31281d4606983371a755c8f25f)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:12:54.101899 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5faa72be-3ad8-476f-b481-9a1f98396d9a admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:12:54.291317 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-5faa72be-3ad8-476f-b481-9a1f98396d9a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5782cf31281d4606983371a755c8f25f'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:12:54.294902 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-5faa72be-3ad8-476f-b481-9a1f98396d9a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5782cf31281d4606983371a755c8f25f'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:12:54.377133 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-5faa72be-3ad8-476f-b481-9a1f98396d9a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5782cf31281d4606983371a755c8f25f'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:12:54.377760 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-5faa72be-3ad8-476f-b481-9a1f98396d9a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5782cf31281d4606983371a755c8f25f'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:12:54.385487 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-5faa72be-3ad8-476f-b481-9a1f98396d9a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5782cf31281d4606983371a755c8f25f'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:12:54.398729 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:12:54.403838 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1433/2842] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 24 10:12:54 2022] DELETE /identity/v3/users/5782cf31281d4606983371a755c8f25f => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:12:54.841497 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933e2beb-cbbc-4251-b7db-d1afee15a2cc admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:12:54.841497 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-933e2beb-cbbc-4251-b7db-d1afee15a2cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:12:54.854504 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933e2beb-cbbc-4251-b7db-d1afee15a2cc admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:12:54.911528 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6796087-115d-4c57-9a53-90ea9511695a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:12:54.912190 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6796087-115d-4c57-9a53-90ea9511695a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:12:54.912473 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6796087-115d-4c57-9a53-90ea9511695a admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:12:54.912748 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6796087-115d-4c57-9a53-90ea9511695a admin admin] PATH_INFO: `/v3/projects/350a545dafcf4716a7771d2638e39325` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:12:54.913870 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6796087-115d-4c57-9a53-90ea9511695a admin admin] RBAC: Authorizing `identity:delete_project(project_id=350a545dafcf4716a7771d2638e39325)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:12:54.915806 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6796087-115d-4c57-9a53-90ea9511695a admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:12:55.011119 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:12:55.027925 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-c6796087-115d-4c57-9a53-90ea9511695a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '350a545dafcf4716a7771d2638e39325', 'request_id': 'req-c6796087-115d-4c57-9a53-90ea9511695a'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:12:55.034084 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-c6796087-115d-4c57-9a53-90ea9511695a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '350a545dafcf4716a7771d2638e39325', 'request_id': 'req-c6796087-115d-4c57-9a53-90ea9511695a'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:12:55.039549 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-c6796087-115d-4c57-9a53-90ea9511695a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '350a545dafcf4716a7771d2638e39325', 'request_id': 'req-c6796087-115d-4c57-9a53-90ea9511695a'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:12:55.134696 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-c6796087-115d-4c57-9a53-90ea9511695a admin admin] The token cache is being invalidate because project 350a545dafcf4716a7771d2638e39325 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:12:55.139082 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-c6796087-115d-4c57-9a53-90ea9511695a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:12:55.139082 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-c6796087-115d-4c57-9a53-90ea9511695a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:12:55.139082 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1410/2843] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 24 10:12:54 2022] DELETE /identity/v3/projects/350a545dafcf4716a7771d2638e39325 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:12:55.142847 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d04aa87-e0a1-4e2c-99fc-0ae1ce32a250 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:12:55.143291 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d04aa87-e0a1-4e2c-99fc-0ae1ce32a250 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:12:55.143405 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d04aa87-e0a1-4e2c-99fc-0ae1ce32a250 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:12:55.162380 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-0d04aa87-e0a1-4e2c-99fc-0ae1ce32a250 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:12:55.163609 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-0d04aa87-e0a1-4e2c-99fc-0ae1ce32a250 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:12:55.222892 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1434/2844] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 10:12:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:12:55.230512 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2d106c-cff9-4789-a39b-59119259771b None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:12:55.230796 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2d106c-cff9-4789-a39b-59119259771b None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:12:55.231071 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2d106c-cff9-4789-a39b-59119259771b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:12:55.250805 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-5f2d106c-cff9-4789-a39b-59119259771b None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:12:55.252547 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-5f2d106c-cff9-4789-a39b-59119259771b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:12:55.260465 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1411/2845] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 10:12:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:12:55.264684 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d04aa87-e0a1-4e2c-99fc-0ae1ce32a250 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:12:55.274918 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d04aa87-e0a1-4e2c-99fc-0ae1ce32a250 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:12:55.279528 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccdcdef-355a-4956-bf1a-99cdc8a155b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:12:55.280237 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccdcdef-355a-4956-bf1a-99cdc8a155b1 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:12:55.280443 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccdcdef-355a-4956-bf1a-99cdc8a155b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:12:55.280721 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccdcdef-355a-4956-bf1a-99cdc8a155b1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:12:55.281394 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccdcdef-355a-4956-bf1a-99cdc8a155b1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:12:55.281699 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccdcdef-355a-4956-bf1a-99cdc8a155b1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:12:55.283288 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccdcdef-355a-4956-bf1a-99cdc8a155b1 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:12:55.294593 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1435/2846] 10.222.0.16 () {58 vars in 1153 bytes} [Wed Aug 24 10:12:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 10:12:55.299365 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2d106c-cff9-4789-a39b-59119259771b None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:12:55.308527 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2d106c-cff9-4789-a39b-59119259771b None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:12:55.313347 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3431a9af-d8a8-4ec9-a797-f96cacaafe27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:12:55.314563 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3431a9af-d8a8-4ec9-a797-f96cacaafe27 admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:12:55.314839 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3431a9af-d8a8-4ec9-a797-f96cacaafe27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:12:55.315106 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3431a9af-d8a8-4ec9-a797-f96cacaafe27 admin admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:12:55.315779 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3431a9af-d8a8-4ec9-a797-f96cacaafe27 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:12:55.317681 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3431a9af-d8a8-4ec9-a797-f96cacaafe27 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:12:55.430653 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1412/2847] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 24 10:12:55 2022] POST /identity/v3/projects => generated 407 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:12:55.434896 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccdcdef-355a-4956-bf1a-99cdc8a155b1 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:12:55.445908 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccdcdef-355a-4956-bf1a-99cdc8a155b1 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:12:55.501658 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775f5270-9907-4e1f-b05c-836c57647556 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:12:55.502255 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775f5270-9907-4e1f-b05c-836c57647556 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:12:55.502534 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775f5270-9907-4e1f-b05c-836c57647556 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:12:55.502802 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775f5270-9907-4e1f-b05c-836c57647556 admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:12:55.503437 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775f5270-9907-4e1f-b05c-836c57647556 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:12:55.505345 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775f5270-9907-4e1f-b05c-836c57647556 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:12:55.649157 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1436/2848] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 24 10:12:55 2022] POST /identity/v3/users => generated 304 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:12:55.653395 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3431a9af-d8a8-4ec9-a797-f96cacaafe27 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:12:55.664029 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3431a9af-d8a8-4ec9-a797-f96cacaafe27 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:12:55.668669 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145bda3a-eaa7-43b8-b985-4acba6a1bb73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:12:55.669294 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145bda3a-eaa7-43b8-b985-4acba6a1bb73 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:12:55.672627 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145bda3a-eaa7-43b8-b985-4acba6a1bb73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:12:55.672627 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145bda3a-eaa7-43b8-b985-4acba6a1bb73 admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:12:55.672627 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145bda3a-eaa7-43b8-b985-4acba6a1bb73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:12:55.672627 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145bda3a-eaa7-43b8-b985-4acba6a1bb73 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:12:55.683669 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1413/2849] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 24 10:12:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:12:55.687861 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775f5270-9907-4e1f-b05c-836c57647556 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:12:55.698080 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775f5270-9907-4e1f-b05c-836c57647556 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:12:55.702992 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5e841c-a591-41ed-bcb4-cf20e9bd6ec3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:12:55.703610 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5e841c-a591-41ed-bcb4-cf20e9bd6ec3 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:12:55.703898 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5e841c-a591-41ed-bcb4-cf20e9bd6ec3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:12:55.704172 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5e841c-a591-41ed-bcb4-cf20e9bd6ec3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:12:55.704823 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5e841c-a591-41ed-bcb4-cf20e9bd6ec3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:12:55.706495 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5e841c-a591-41ed-bcb4-cf20e9bd6ec3 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:12:55.717403 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1437/2850] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 24 10:12:55 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:12:55.723040 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145bda3a-eaa7-43b8-b985-4acba6a1bb73 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:12:55.733982 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145bda3a-eaa7-43b8-b985-4acba6a1bb73 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:12:55.739210 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a6825a-f57c-4ea7-b331-1834b01b577f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:12:55.739880 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a6825a-f57c-4ea7-b331-1834b01b577f admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:12:55.740159 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a6825a-f57c-4ea7-b331-1834b01b577f admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:12:55.740429 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a6825a-f57c-4ea7-b331-1834b01b577f admin admin] PATH_INFO: `/v3/projects/b44f16fbfdfe4ffba2deb0f5892f12ac/users/c2551c1354584b55a3d4bc676655f640/roles/136cdf78e00a426997dd4e37faff489b` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:12:55.755376 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a6825a-f57c-4ea7-b331-1834b01b577f admin admin] RBAC: Authorizing `identity:create_grant(project_id=b44f16fbfdfe4ffba2deb0f5892f12ac, user_id=c2551c1354584b55a3d4bc676655f640, role_id=136cdf78e00a426997dd4e37faff489b)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:12:55.758022 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a6825a-f57c-4ea7-b331-1834b01b577f admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:12:55.845801 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1414/2851] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 24 10:12:55 2022] PUT /identity/v3/projects/b44f16fbfdfe4ffba2deb0f5892f12ac/users/c2551c1354584b55a3d4bc676655f640/roles/136cdf78e00a426997dd4e37faff489b => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:12:55.856932 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db073107-0af7-41ce-8ece-cc2a9d0955b8 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:12:55.857268 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db073107-0af7-41ce-8ece-cc2a9d0955b8 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:12:55.857546 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db073107-0af7-41ce-8ece-cc2a9d0955b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:12:55.884915 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-db073107-0af7-41ce-8ece-cc2a9d0955b8 None None] MFA Rules not processed for user `c2551c1354584b55a3d4bc676655f640`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:12:55.886117 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-db073107-0af7-41ce-8ece-cc2a9d0955b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:12:55.939129 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:12:55.948636 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1438/2852] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 10:12:55 2022] POST /identity/v3/auth/tokens => generated 3284 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:12:55.953155 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a6825a-f57c-4ea7-b331-1834b01b577f admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:12:56.015791 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a6825a-f57c-4ea7-b331-1834b01b577f admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:12:56.021018 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb3fb18-5c09-4435-9d70-820f9fe05487 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:12:56.021582 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb3fb18-5c09-4435-9d70-820f9fe05487 admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:12:56.021858 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb3fb18-5c09-4435-9d70-820f9fe05487 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:12:56.022133 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb3fb18-5c09-4435-9d70-820f9fe05487 admin admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:12:56.022764 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb3fb18-5c09-4435-9d70-820f9fe05487 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:12:56.024732 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb3fb18-5c09-4435-9d70-820f9fe05487 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:12:56.204618 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1415/2853] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 24 10:12:55 2022] POST /identity/v3/projects => generated 407 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:12:56.208308 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db073107-0af7-41ce-8ece-cc2a9d0955b8 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:12:56.218833 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db073107-0af7-41ce-8ece-cc2a9d0955b8 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:12:56.273848 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5507eb86-ee54-4eb3-8db3-cb17db8ecebc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:12:56.274446 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5507eb86-ee54-4eb3-8db3-cb17db8ecebc admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:12:56.274726 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5507eb86-ee54-4eb3-8db3-cb17db8ecebc admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:12:56.274998 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5507eb86-ee54-4eb3-8db3-cb17db8ecebc admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:12:56.275650 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5507eb86-ee54-4eb3-8db3-cb17db8ecebc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:12:56.277558 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5507eb86-ee54-4eb3-8db3-cb17db8ecebc admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:12:56.432681 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1439/2854] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 24 10:12:56 2022] POST /identity/v3/users => generated 310 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:12:56.436763 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb3fb18-5c09-4435-9d70-820f9fe05487 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:12:56.447028 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb3fb18-5c09-4435-9d70-820f9fe05487 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:12:56.451599 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9826c8cf-f7a2-4232-b352-f30378a629cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:12:56.452205 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9826c8cf-f7a2-4232-b352-f30378a629cd admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:12:56.452492 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9826c8cf-f7a2-4232-b352-f30378a629cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:12:56.452771 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9826c8cf-f7a2-4232-b352-f30378a629cd admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:12:56.453413 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9826c8cf-f7a2-4232-b352-f30378a629cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:12:56.455095 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9826c8cf-f7a2-4232-b352-f30378a629cd admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:12:56.465640 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1416/2855] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 24 10:12:56 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:12:56.470345 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5507eb86-ee54-4eb3-8db3-cb17db8ecebc admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:12:56.480250 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5507eb86-ee54-4eb3-8db3-cb17db8ecebc admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:12:56.484772 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0917df-611f-42fc-8105-33579775f841 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:12:56.486002 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0917df-611f-42fc-8105-33579775f841 admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:12:56.486281 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0917df-611f-42fc-8105-33579775f841 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:12:56.486553 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0917df-611f-42fc-8105-33579775f841 admin admin] PATH_INFO: `/v3/projects/262538bdb0804a6aa8cf96608fd7bf1d/users/8de2225a125a4b129151102cac670775/roles/15f634d4a2d247ddab90c458e5e1c141` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:12:56.500708 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0917df-611f-42fc-8105-33579775f841 admin admin] RBAC: Authorizing `identity:create_grant(project_id=262538bdb0804a6aa8cf96608fd7bf1d, user_id=8de2225a125a4b129151102cac670775, role_id=15f634d4a2d247ddab90c458e5e1c141)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:12:56.503338 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0917df-611f-42fc-8105-33579775f841 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:12:56.598135 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1440/2856] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 24 10:12:56 2022] PUT /identity/v3/projects/262538bdb0804a6aa8cf96608fd7bf1d/users/8de2225a125a4b129151102cac670775/roles/15f634d4a2d247ddab90c458e5e1c141 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:12:56.602149 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9826c8cf-f7a2-4232-b352-f30378a629cd admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:12:56.612663 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9826c8cf-f7a2-4232-b352-f30378a629cd admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:12:56.632680 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:12:56.667198 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5380d723-e84c-48bc-890a-40781a9e8438 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:12:56.668013 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5380d723-e84c-48bc-890a-40781a9e8438 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:12:56.668294 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5380d723-e84c-48bc-890a-40781a9e8438 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:12:56.668563 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5380d723-e84c-48bc-890a-40781a9e8438 admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:12:56.669188 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5380d723-e84c-48bc-890a-40781a9e8438 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:12:56.670766 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5380d723-e84c-48bc-890a-40781a9e8438 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:12:56.682244 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1417/2857] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 24 10:12:56 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:12:56.685161 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0917df-611f-42fc-8105-33579775f841 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:12:56.695616 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0917df-611f-42fc-8105-33579775f841 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:12:56.700738 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac49389c-f2d9-4e4f-8dd9-dd8822f6f240 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:12:56.701387 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac49389c-f2d9-4e4f-8dd9-dd8822f6f240 admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:12:56.701667 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac49389c-f2d9-4e4f-8dd9-dd8822f6f240 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:12:56.701941 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac49389c-f2d9-4e4f-8dd9-dd8822f6f240 admin admin] PATH_INFO: `/v3/projects/262538bdb0804a6aa8cf96608fd7bf1d/users/8de2225a125a4b129151102cac670775/roles/15f634d4a2d247ddab90c458e5e1c141` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:12:56.703975 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac49389c-f2d9-4e4f-8dd9-dd8822f6f240 admin admin] RBAC: Authorizing `identity:create_grant(project_id=262538bdb0804a6aa8cf96608fd7bf1d, user_id=8de2225a125a4b129151102cac670775, role_id=15f634d4a2d247ddab90c458e5e1c141)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:12:56.706654 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac49389c-f2d9-4e4f-8dd9-dd8822f6f240 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:12:56.716885 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1441/2858] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 24 10:12:56 2022] PUT /identity/v3/projects/262538bdb0804a6aa8cf96608fd7bf1d/users/8de2225a125a4b129151102cac670775/roles/15f634d4a2d247ddab90c458e5e1c141 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:12:56.727553 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97eb4a33-b801-4247-809e-ab08bb75c016 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:12:56.727931 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97eb4a33-b801-4247-809e-ab08bb75c016 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:12:56.728273 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97eb4a33-b801-4247-809e-ab08bb75c016 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:12:56.755553 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-97eb4a33-b801-4247-809e-ab08bb75c016 None None] MFA Rules not processed for user `8de2225a125a4b129151102cac670775`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:12:56.756767 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-97eb4a33-b801-4247-809e-ab08bb75c016 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:12:56.823440 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1418/2859] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 10:12:56 2022] POST /identity/v3/auth/tokens => generated 3351 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:12:56.834438 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac49389c-f2d9-4e4f-8dd9-dd8822f6f240 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:12:56.836179 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-ac49389c-f2d9-4e4f-8dd9-dd8822f6f240 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:12:56.893537 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac49389c-f2d9-4e4f-8dd9-dd8822f6f240 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:12:56.897213 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363651ba-ab59-4377-8f5b-ab5267279278 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c32b83577c46488906fe409a790b24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:12:56.897816 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363651ba-ab59-4377-8f5b-ab5267279278 service cinder] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:12:56.898096 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363651ba-ab59-4377-8f5b-ab5267279278 service cinder] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:12:56.898368 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363651ba-ab59-4377-8f5b-ab5267279278 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:12:56.907839 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363651ba-ab59-4377-8f5b-ab5267279278 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:12:56.909387 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363651ba-ab59-4377-8f5b-ab5267279278 service cinder] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:12:56.918827 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1442/2860] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 10:12:56 2022] GET /identity/v3/auth/tokens => generated 3351 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:12:57.268596 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97eb4a33-b801-4247-809e-ab08bb75c016 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:12:57.278253 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97eb4a33-b801-4247-809e-ab08bb75c016 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:12:57.281849 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fbda58-2c55-45b3-bcf9-204cd9069185 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c32b83577c46488906fe409a790b24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:12:57.282972 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fbda58-2c55-45b3-bcf9-204cd9069185 service cinder] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:12:57.283250 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fbda58-2c55-45b3-bcf9-204cd9069185 service cinder] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:12:57.283519 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fbda58-2c55-45b3-bcf9-204cd9069185 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:12:57.339556 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fbda58-2c55-45b3-bcf9-204cd9069185 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:12:57.341041 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fbda58-2c55-45b3-bcf9-204cd9069185 service cinder] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:12:57.350688 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1419/2861] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 10:12:57 2022] GET /identity/v3/auth/tokens => generated 3284 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:13:06.914669 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=73,keystone:INSERT=5 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:13:07.346592 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=80 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:13:10.268125 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363651ba-ab59-4377-8f5b-ab5267279278 service cinder] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:13:10.329624 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363651ba-ab59-4377-8f5b-ab5267279278 service cinder] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:13:10.334517 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b147a3-2d77-4d27-a0e8-ccc48b134fb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:13:10.335817 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b147a3-2d77-4d27-a0e8-ccc48b134fb0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:13:10.336104 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b147a3-2d77-4d27-a0e8-ccc48b134fb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:13:10.336377 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b147a3-2d77-4d27-a0e8-ccc48b134fb0 admin admin] PATH_INFO: `/v3/users/c2551c1354584b55a3d4bc676655f640` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:13:10.337506 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b147a3-2d77-4d27-a0e8-ccc48b134fb0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c2551c1354584b55a3d4bc676655f640)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:13:10.339412 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b147a3-2d77-4d27-a0e8-ccc48b134fb0 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:13:10.556593 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-d5b147a3-2d77-4d27-a0e8-ccc48b134fb0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c2551c1354584b55a3d4bc676655f640'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:13:10.557250 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-d5b147a3-2d77-4d27-a0e8-ccc48b134fb0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c2551c1354584b55a3d4bc676655f640'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:13:10.631599 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-d5b147a3-2d77-4d27-a0e8-ccc48b134fb0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c2551c1354584b55a3d4bc676655f640'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:13:10.632766 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-d5b147a3-2d77-4d27-a0e8-ccc48b134fb0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c2551c1354584b55a3d4bc676655f640'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:13:10.639939 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-d5b147a3-2d77-4d27-a0e8-ccc48b134fb0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c2551c1354584b55a3d4bc676655f640'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:13:10.657426 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1443/2862] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 24 10:13:10 2022] DELETE /identity/v3/users/c2551c1354584b55a3d4bc676655f640 => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:13:10.662130 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea672b59-df76-401c-a60c-d196a692deef None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:13:10.662608 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea672b59-df76-401c-a60c-d196a692deef None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:13:10.662716 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea672b59-df76-401c-a60c-d196a692deef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:13:10.682625 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-ea672b59-df76-401c-a60c-d196a692deef None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:13:10.683854 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-ea672b59-df76-401c-a60c-d196a692deef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:13:10.742131 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1420/2863] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 10:13:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:13:10.751897 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b147a3-2d77-4d27-a0e8-ccc48b134fb0 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:13:10.753315 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-d5b147a3-2d77-4d27-a0e8-ccc48b134fb0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:13:10.809031 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b147a3-2d77-4d27-a0e8-ccc48b134fb0 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:13:10.812569 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c202b683-1463-426f-9c76-a2ab708e1d61 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e5260e798404e8bb12b25ca8ea97052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:13:10.813171 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c202b683-1463-426f-9c76-a2ab708e1d61 service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:13:10.813450 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c202b683-1463-426f-9c76-a2ab708e1d61 service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:13:10.813724 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c202b683-1463-426f-9c76-a2ab708e1d61 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:13:10.823145 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c202b683-1463-426f-9c76-a2ab708e1d61 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:13:10.824581 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c202b683-1463-426f-9c76-a2ab708e1d61 service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:13:10.834399 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1444/2864] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 10:13:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:13:11.267196 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea672b59-df76-401c-a60c-d196a692deef None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:13:11.268839 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-ea672b59-df76-401c-a60c-d196a692deef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:13:11.281951 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea672b59-df76-401c-a60c-d196a692deef None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:13:11.286502 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83250f35-3d7c-46fa-9301-2fc232993e19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:13:11.287230 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83250f35-3d7c-46fa-9301-2fc232993e19 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:13:11.287442 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83250f35-3d7c-46fa-9301-2fc232993e19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:13:11.287750 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83250f35-3d7c-46fa-9301-2fc232993e19 admin admin] PATH_INFO: `/v3/projects/b44f16fbfdfe4ffba2deb0f5892f12ac` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:13:11.288975 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83250f35-3d7c-46fa-9301-2fc232993e19 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b44f16fbfdfe4ffba2deb0f5892f12ac)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:13:11.290901 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83250f35-3d7c-46fa-9301-2fc232993e19 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:13:11.393393 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-83250f35-3d7c-46fa-9301-2fc232993e19 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b44f16fbfdfe4ffba2deb0f5892f12ac', 'request_id': 'req-83250f35-3d7c-46fa-9301-2fc232993e19'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:13:11.399481 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-83250f35-3d7c-46fa-9301-2fc232993e19 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b44f16fbfdfe4ffba2deb0f5892f12ac', 'request_id': 'req-83250f35-3d7c-46fa-9301-2fc232993e19'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:13:11.405043 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-83250f35-3d7c-46fa-9301-2fc232993e19 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b44f16fbfdfe4ffba2deb0f5892f12ac', 'request_id': 'req-83250f35-3d7c-46fa-9301-2fc232993e19'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:13:11.471352 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-83250f35-3d7c-46fa-9301-2fc232993e19 admin admin] The token cache is being invalidate because project b44f16fbfdfe4ffba2deb0f5892f12ac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:13:11.471664 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-83250f35-3d7c-46fa-9301-2fc232993e19 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:13:11.472281 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-83250f35-3d7c-46fa-9301-2fc232993e19 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:13:11.473849 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1421/2865] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 24 10:13:11 2022] DELETE /identity/v3/projects/b44f16fbfdfe4ffba2deb0f5892f12ac => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:13:11.477864 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c202b683-1463-426f-9c76-a2ab708e1d61 service neutron] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:13:11.487676 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c202b683-1463-426f-9c76-a2ab708e1d61 service neutron] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:13:11.505996 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:13:11.543759 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045bbf57-6049-4a6b-a5e0-ba08cce659d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:13:11.544391 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045bbf57-6049-4a6b-a5e0-ba08cce659d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:13:11.544671 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045bbf57-6049-4a6b-a5e0-ba08cce659d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:13:11.544945 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045bbf57-6049-4a6b-a5e0-ba08cce659d2 admin admin] PATH_INFO: `/v3/users/8de2225a125a4b129151102cac670775` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:13:11.546067 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045bbf57-6049-4a6b-a5e0-ba08cce659d2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8de2225a125a4b129151102cac670775)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:13:11.548028 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045bbf57-6049-4a6b-a5e0-ba08cce659d2 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:13:11.758934 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-045bbf57-6049-4a6b-a5e0-ba08cce659d2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8de2225a125a4b129151102cac670775'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:13:11.759513 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-045bbf57-6049-4a6b-a5e0-ba08cce659d2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8de2225a125a4b129151102cac670775'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:13:11.825910 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-045bbf57-6049-4a6b-a5e0-ba08cce659d2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8de2225a125a4b129151102cac670775'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:13:11.826544 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-045bbf57-6049-4a6b-a5e0-ba08cce659d2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8de2225a125a4b129151102cac670775'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:13:11.833674 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-045bbf57-6049-4a6b-a5e0-ba08cce659d2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8de2225a125a4b129151102cac670775'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:13:11.851671 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1445/2866] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 24 10:13:11 2022] DELETE /identity/v3/users/8de2225a125a4b129151102cac670775 => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:13:12.250753 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83250f35-3d7c-46fa-9301-2fc232993e19 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:13:12.252420 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-83250f35-3d7c-46fa-9301-2fc232993e19 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:13:12.266996 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83250f35-3d7c-46fa-9301-2fc232993e19 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:13:12.324670 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fd7e82-057e-4e43-bb91-733a9225cb04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:13:12.325304 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fd7e82-057e-4e43-bb91-733a9225cb04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:13:12.325588 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fd7e82-057e-4e43-bb91-733a9225cb04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:13:12.325909 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fd7e82-057e-4e43-bb91-733a9225cb04 admin admin] PATH_INFO: `/v3/projects/262538bdb0804a6aa8cf96608fd7bf1d` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:13:12.327030 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fd7e82-057e-4e43-bb91-733a9225cb04 admin admin] RBAC: Authorizing `identity:delete_project(project_id=262538bdb0804a6aa8cf96608fd7bf1d)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:13:12.328951 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fd7e82-057e-4e43-bb91-733a9225cb04 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:13:12.439199 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:13:12.456066 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-06fd7e82-057e-4e43-bb91-733a9225cb04 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '262538bdb0804a6aa8cf96608fd7bf1d', 'request_id': 'req-06fd7e82-057e-4e43-bb91-733a9225cb04'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:13:12.462819 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-06fd7e82-057e-4e43-bb91-733a9225cb04 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '262538bdb0804a6aa8cf96608fd7bf1d', 'request_id': 'req-06fd7e82-057e-4e43-bb91-733a9225cb04'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:13:12.468320 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-06fd7e82-057e-4e43-bb91-733a9225cb04 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '262538bdb0804a6aa8cf96608fd7bf1d', 'request_id': 'req-06fd7e82-057e-4e43-bb91-733a9225cb04'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:13:12.565046 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-06fd7e82-057e-4e43-bb91-733a9225cb04 admin admin] The token cache is being invalidate because project 262538bdb0804a6aa8cf96608fd7bf1d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:13:12.565338 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-06fd7e82-057e-4e43-bb91-733a9225cb04 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:13:12.565956 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-06fd7e82-057e-4e43-bb91-733a9225cb04 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:13:12.567434 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1422/2867] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 24 10:13:12 2022] DELETE /identity/v3/projects/262538bdb0804a6aa8cf96608fd7bf1d => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:13:12.572735 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2083296e-2d70-4191-9cc2-3f07e9ecacc3 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:13:12.573226 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2083296e-2d70-4191-9cc2-3f07e9ecacc3 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:13:12.573340 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2083296e-2d70-4191-9cc2-3f07e9ecacc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:13:12.593049 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-2083296e-2d70-4191-9cc2-3f07e9ecacc3 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:13:12.594251 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-2083296e-2d70-4191-9cc2-3f07e9ecacc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:13:12.651524 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1446/2868] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 10:13:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:13:12.659262 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43899b7d-c3c0-4742-b8c2-11772b069a9c None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:13:12.659555 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43899b7d-c3c0-4742-b8c2-11772b069a9c None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:13:12.659869 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43899b7d-c3c0-4742-b8c2-11772b069a9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:13:12.680231 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-43899b7d-c3c0-4742-b8c2-11772b069a9c None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:13:12.681435 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-43899b7d-c3c0-4742-b8c2-11772b069a9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:13:12.690271 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1423/2869] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 10:13:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:13:12.694969 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2083296e-2d70-4191-9cc2-3f07e9ecacc3 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:13:12.708265 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2083296e-2d70-4191-9cc2-3f07e9ecacc3 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:13:12.713406 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db626b1-f00d-4e3b-a453-933b70b86cd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:13:12.714008 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db626b1-f00d-4e3b-a453-933b70b86cd2 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:13:12.714287 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db626b1-f00d-4e3b-a453-933b70b86cd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:13:12.714560 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db626b1-f00d-4e3b-a453-933b70b86cd2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:13:12.715205 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db626b1-f00d-4e3b-a453-933b70b86cd2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81602) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:13:12.715508 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db626b1-f00d-4e3b-a453-933b70b86cd2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:13:12.717142 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db626b1-f00d-4e3b-a453-933b70b86cd2 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:13:12.728611 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1447/2870] 10.222.0.16 () {58 vars in 1153 bytes} [Wed Aug 24 10:13:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:13:12.734248 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43899b7d-c3c0-4742-b8c2-11772b069a9c None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:13:12.743491 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43899b7d-c3c0-4742-b8c2-11772b069a9c None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:13:12.748344 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f0bee2-83fd-4547-857c-8e12b53dddf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:13:12.748946 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f0bee2-83fd-4547-857c-8e12b53dddf0 admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:13:12.749254 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f0bee2-83fd-4547-857c-8e12b53dddf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:13:12.749525 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f0bee2-83fd-4547-857c-8e12b53dddf0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:13:12.750152 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f0bee2-83fd-4547-857c-8e12b53dddf0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:13:12.752165 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f0bee2-83fd-4547-857c-8e12b53dddf0 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:13:12.841309 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1424/2871] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 24 10:13:12 2022] POST /identity/v3/projects => generated 379 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 10:13:12.845930 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db626b1-f00d-4e3b-a453-933b70b86cd2 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:13:12.855920 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db626b1-f00d-4e3b-a453-933b70b86cd2 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:13:12.881610 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:13:12.912010 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8bb4c9-480c-4622-9828-994222fef0e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:13:12.912621 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8bb4c9-480c-4622-9828-994222fef0e9 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:13:12.913250 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8bb4c9-480c-4622-9828-994222fef0e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:13:12.913645 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8bb4c9-480c-4622-9828-994222fef0e9 admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:13:12.914312 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8bb4c9-480c-4622-9828-994222fef0e9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:13:12.916363 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8bb4c9-480c-4622-9828-994222fef0e9 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:13:12.977032 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1448/2872] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 24 10:13:12 2022] POST /identity/v3/users => generated 290 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:13:12.981234 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f0bee2-83fd-4547-857c-8e12b53dddf0 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:13:12.991477 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f0bee2-83fd-4547-857c-8e12b53dddf0 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:13:12.996518 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef957244-dee9-4b73-b458-bb205b2d1779 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:13:12.997110 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef957244-dee9-4b73-b458-bb205b2d1779 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:13:12.997386 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef957244-dee9-4b73-b458-bb205b2d1779 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:13:12.997655 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef957244-dee9-4b73-b458-bb205b2d1779 admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:13:12.998284 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef957244-dee9-4b73-b458-bb205b2d1779 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:13:12.999951 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef957244-dee9-4b73-b458-bb205b2d1779 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:13:13.010535 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1425/2873] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 24 10:13:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:13:13.014391 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8bb4c9-480c-4622-9828-994222fef0e9 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:13:13.024288 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8bb4c9-480c-4622-9828-994222fef0e9 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:13:13.029427 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51a1b7c-34d5-450a-85e3-7c02a2d9cc12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:13:13.029553 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51a1b7c-34d5-450a-85e3-7c02a2d9cc12 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:13:13.029804 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51a1b7c-34d5-450a-85e3-7c02a2d9cc12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:13:13.030079 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51a1b7c-34d5-450a-85e3-7c02a2d9cc12 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:13:13.030724 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51a1b7c-34d5-450a-85e3-7c02a2d9cc12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:13:13.032358 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51a1b7c-34d5-450a-85e3-7c02a2d9cc12 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:13:13.042067 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1449/2874] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 24 10:13:13 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 10:13:13.046101 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef957244-dee9-4b73-b458-bb205b2d1779 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:13:13.056333 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef957244-dee9-4b73-b458-bb205b2d1779 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:13:13.060946 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bc9d23-6da6-4eb7-837d-28b6662824d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:13:13.061587 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bc9d23-6da6-4eb7-837d-28b6662824d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:13:13.061863 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bc9d23-6da6-4eb7-837d-28b6662824d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:13:13.062132 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bc9d23-6da6-4eb7-837d-28b6662824d2 admin admin] PATH_INFO: `/v3/projects/2299f136823d4a0b81e5279ab66b15b5/users/3dcce286c4dd4168acdee839a7b4b295/roles/136cdf78e00a426997dd4e37faff489b` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:13:13.076916 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bc9d23-6da6-4eb7-837d-28b6662824d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2299f136823d4a0b81e5279ab66b15b5, user_id=3dcce286c4dd4168acdee839a7b4b295, role_id=136cdf78e00a426997dd4e37faff489b)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:13:13.079612 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bc9d23-6da6-4eb7-837d-28b6662824d2 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:13:13.145959 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1426/2875] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 24 10:13:13 2022] PUT /identity/v3/projects/2299f136823d4a0b81e5279ab66b15b5/users/3dcce286c4dd4168acdee839a7b4b295/roles/136cdf78e00a426997dd4e37faff489b => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 10:13:13.155807 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0552f843-e379-4852-adb3-1529e089a802 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:13:13.156096 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0552f843-e379-4852-adb3-1529e089a802 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:13:13.156374 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0552f843-e379-4852-adb3-1529e089a802 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:13:13.184904 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-0552f843-e379-4852-adb3-1529e089a802 None None] MFA Rules not processed for user `3dcce286c4dd4168acdee839a7b4b295`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:13:13.186141 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-0552f843-e379-4852-adb3-1529e089a802 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:13:13.248876 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1450/2876] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 10:13:13 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:13:13.253649 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bc9d23-6da6-4eb7-837d-28b6662824d2 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:13:13.314912 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bc9d23-6da6-4eb7-837d-28b6662824d2 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:13:13.319744 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff1b8cc-6b2c-4ada-b0a7-1836fc7ac978 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:13:13.320372 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff1b8cc-6b2c-4ada-b0a7-1836fc7ac978 admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:13:13.320655 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff1b8cc-6b2c-4ada-b0a7-1836fc7ac978 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:13:13.320923 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff1b8cc-6b2c-4ada-b0a7-1836fc7ac978 admin admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:13:13.321550 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff1b8cc-6b2c-4ada-b0a7-1836fc7ac978 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:13:13.323479 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff1b8cc-6b2c-4ada-b0a7-1836fc7ac978 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:13:13.407002 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1427/2877] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 24 10:13:13 2022] POST /identity/v3/projects => generated 379 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:13:13.411037 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0552f843-e379-4852-adb3-1529e089a802 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:13:13.422082 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0552f843-e379-4852-adb3-1529e089a802 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:13:13.477796 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe72f2f3-cf0b-4a79-829a-dd6546fbc607 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:13:13.478397 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe72f2f3-cf0b-4a79-829a-dd6546fbc607 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:13:13.478676 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe72f2f3-cf0b-4a79-829a-dd6546fbc607 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:13:13.478946 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe72f2f3-cf0b-4a79-829a-dd6546fbc607 admin admin] PATH_INFO: `/v3/users` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:13:13.479558 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe72f2f3-cf0b-4a79-829a-dd6546fbc607 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:13:13.481478 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe72f2f3-cf0b-4a79-829a-dd6546fbc607 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:13:13.580368 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1451/2878] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 24 10:13:13 2022] POST /identity/v3/users => generated 296 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:13:13.584367 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff1b8cc-6b2c-4ada-b0a7-1836fc7ac978 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:13:13.595238 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff1b8cc-6b2c-4ada-b0a7-1836fc7ac978 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:13:13.599820 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf840052-3c3a-42c7-94f3-c720374ce2f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:13:13.600415 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf840052-3c3a-42c7-94f3-c720374ce2f1 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:13:13.600695 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf840052-3c3a-42c7-94f3-c720374ce2f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:13:13.601607 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf840052-3c3a-42c7-94f3-c720374ce2f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:13:13.602273 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf840052-3c3a-42c7-94f3-c720374ce2f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:13:13.603887 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf840052-3c3a-42c7-94f3-c720374ce2f1 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:13:13.614073 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1428/2879] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 24 10:13:13 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:13:13.618243 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe72f2f3-cf0b-4a79-829a-dd6546fbc607 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:13:13.628557 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe72f2f3-cf0b-4a79-829a-dd6546fbc607 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:13:13.633377 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07402bf6-e4b0-4b9c-9479-a3dc9517ee99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:13:13.634028 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07402bf6-e4b0-4b9c-9479-a3dc9517ee99 admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:13:13.634309 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07402bf6-e4b0-4b9c-9479-a3dc9517ee99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:13:13.634583 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07402bf6-e4b0-4b9c-9479-a3dc9517ee99 admin admin] PATH_INFO: `/v3/projects/7ce8b54d7d7c47119af3ed2864882be9/users/a83ffb4ee13743a6b9639297a7631bed/roles/15f634d4a2d247ddab90c458e5e1c141` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:13:13.650586 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07402bf6-e4b0-4b9c-9479-a3dc9517ee99 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ce8b54d7d7c47119af3ed2864882be9, user_id=a83ffb4ee13743a6b9639297a7631bed, role_id=15f634d4a2d247ddab90c458e5e1c141)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:13:13.653297 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07402bf6-e4b0-4b9c-9479-a3dc9517ee99 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:13:13.789876 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1452/2880] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 24 10:13:13 2022] PUT /identity/v3/projects/7ce8b54d7d7c47119af3ed2864882be9/users/a83ffb4ee13743a6b9639297a7631bed/roles/15f634d4a2d247ddab90c458e5e1c141 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:13:13.793801 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf840052-3c3a-42c7-94f3-c720374ce2f1 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:13:13.803786 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf840052-3c3a-42c7-94f3-c720374ce2f1 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:13:13.823751 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:13:13.861742 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b0787f-6daa-47fa-806d-e5a793873032 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:13:13.862356 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b0787f-6daa-47fa-806d-e5a793873032 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:13:13.862659 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b0787f-6daa-47fa-806d-e5a793873032 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:13:13.862962 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b0787f-6daa-47fa-806d-e5a793873032 admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:13:13.863624 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b0787f-6daa-47fa-806d-e5a793873032 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:13:13.865239 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b0787f-6daa-47fa-806d-e5a793873032 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:13:13.875055 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1429/2881] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 24 10:13:13 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:13:13.879274 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07402bf6-e4b0-4b9c-9479-a3dc9517ee99 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:13:13.884747 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:13:13.890177 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07402bf6-e4b0-4b9c-9479-a3dc9517ee99 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:13:13.894752 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5725fa-2dea-4da1-95d2-a9718b75b499 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:13:13.895399 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5725fa-2dea-4da1-95d2-a9718b75b499 admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:13:13.895710 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5725fa-2dea-4da1-95d2-a9718b75b499 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:13:13.895983 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5725fa-2dea-4da1-95d2-a9718b75b499 admin admin] PATH_INFO: `/v3/projects/7ce8b54d7d7c47119af3ed2864882be9/users/a83ffb4ee13743a6b9639297a7631bed/roles/15f634d4a2d247ddab90c458e5e1c141` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:13:13.898478 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5725fa-2dea-4da1-95d2-a9718b75b499 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ce8b54d7d7c47119af3ed2864882be9, user_id=a83ffb4ee13743a6b9639297a7631bed, role_id=15f634d4a2d247ddab90c458e5e1c141)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:13:13.901330 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5725fa-2dea-4da1-95d2-a9718b75b499 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:13:13.911212 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1453/2882] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 24 10:13:13 2022] PUT /identity/v3/projects/7ce8b54d7d7c47119af3ed2864882be9/users/a83ffb4ee13743a6b9639297a7631bed/roles/15f634d4a2d247ddab90c458e5e1c141 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:13:13.921982 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26ac043-0556-4ce9-b797-91a0e67cc2f1 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:13:13.922365 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26ac043-0556-4ce9-b797-91a0e67cc2f1 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:13:13.922641 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26ac043-0556-4ce9-b797-91a0e67cc2f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:13:13.950228 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-e26ac043-0556-4ce9-b797-91a0e67cc2f1 None None] MFA Rules not processed for user `a83ffb4ee13743a6b9639297a7631bed`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:13:13.951429 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-e26ac043-0556-4ce9-b797-91a0e67cc2f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:13:14.016911 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1430/2883] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 10:13:13 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:13:14.028283 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5725fa-2dea-4da1-95d2-a9718b75b499 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:13:14.033455 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-eb5725fa-2dea-4da1-95d2-a9718b75b499 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:13:14.091076 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5725fa-2dea-4da1-95d2-a9718b75b499 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:13:14.094690 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99ad512-1415-49f1-8388-9aecd9133818 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c32b83577c46488906fe409a790b24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:13:14.095547 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99ad512-1415-49f1-8388-9aecd9133818 service cinder] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:13:14.095925 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99ad512-1415-49f1-8388-9aecd9133818 service cinder] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:13:14.096197 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99ad512-1415-49f1-8388-9aecd9133818 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:13:14.105077 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99ad512-1415-49f1-8388-9aecd9133818 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:13:14.106522 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99ad512-1415-49f1-8388-9aecd9133818 service cinder] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:13:14.115855 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1454/2884] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 10:13:14 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:13:14.652289 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26ac043-0556-4ce9-b797-91a0e67cc2f1 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:13:14.713729 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26ac043-0556-4ce9-b797-91a0e67cc2f1 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:13:14.718438 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fbd97f-25a9-45c6-b913-15f06ef8ff81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:13:14.719065 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fbd97f-25a9-45c6-b913-15f06ef8ff81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:13:14.719346 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fbd97f-25a9-45c6-b913-15f06ef8ff81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:13:14.719646 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fbd97f-25a9-45c6-b913-15f06ef8ff81 admin admin] PATH_INFO: `/v3/users/3dcce286c4dd4168acdee839a7b4b295` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:13:14.721049 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fbd97f-25a9-45c6-b913-15f06ef8ff81 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3dcce286c4dd4168acdee839a7b4b295)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:13:14.722981 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fbd97f-25a9-45c6-b913-15f06ef8ff81 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:13:14.907798 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-30fbd97f-25a9-45c6-b913-15f06ef8ff81 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3dcce286c4dd4168acdee839a7b4b295'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:13:14.908405 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-30fbd97f-25a9-45c6-b913-15f06ef8ff81 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3dcce286c4dd4168acdee839a7b4b295'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:13:14.909532 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:13:14.921473 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-30fbd97f-25a9-45c6-b913-15f06ef8ff81 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3dcce286c4dd4168acdee839a7b4b295'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:13:14.928420 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-30fbd97f-25a9-45c6-b913-15f06ef8ff81 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3dcce286c4dd4168acdee839a7b4b295'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:13:15.009052 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-30fbd97f-25a9-45c6-b913-15f06ef8ff81 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3dcce286c4dd4168acdee839a7b4b295'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:13:15.015857 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1431/2885] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 24 10:13:14 2022] DELETE /identity/v3/users/3dcce286c4dd4168acdee839a7b4b295 => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:13:15.020417 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde4766e-1679-46c3-b2cb-ac1e5dcd6698 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:13:15.020909 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde4766e-1679-46c3-b2cb-ac1e5dcd6698 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:13:15.021028 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde4766e-1679-46c3-b2cb-ac1e5dcd6698 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:13:15.040647 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-bde4766e-1679-46c3-b2cb-ac1e5dcd6698 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:13:15.041865 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-bde4766e-1679-46c3-b2cb-ac1e5dcd6698 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:13:15.100670 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1455/2886] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 10:13:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:13:15.111189 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fbd97f-25a9-45c6-b913-15f06ef8ff81 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:13:15.113518 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-30fbd97f-25a9-45c6-b913-15f06ef8ff81 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:13:15.171873 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fbd97f-25a9-45c6-b913-15f06ef8ff81 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:13:15.175474 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69fb7db-b576-4a8d-a865-cfa41c290529 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e5260e798404e8bb12b25ca8ea97052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:13:15.176109 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69fb7db-b576-4a8d-a865-cfa41c290529 service neutron] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:13:15.176391 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69fb7db-b576-4a8d-a865-cfa41c290529 service neutron] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:13:15.176666 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69fb7db-b576-4a8d-a865-cfa41c290529 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:13:15.186090 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69fb7db-b576-4a8d-a865-cfa41c290529 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:13:15.187514 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69fb7db-b576-4a8d-a865-cfa41c290529 service neutron] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:13:15.197099 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1432/2887] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 10:13:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:13:15.588507 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde4766e-1679-46c3-b2cb-ac1e5dcd6698 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:13:15.589936 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-bde4766e-1679-46c3-b2cb-ac1e5dcd6698 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:13:15.602555 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde4766e-1679-46c3-b2cb-ac1e5dcd6698 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:13:15.607009 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2075c4b-af6d-406b-ab23-3f9aadf06084 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:13:15.607673 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2075c4b-af6d-406b-ab23-3f9aadf06084 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:13:15.607956 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2075c4b-af6d-406b-ab23-3f9aadf06084 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:13:15.608230 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2075c4b-af6d-406b-ab23-3f9aadf06084 admin admin] PATH_INFO: `/v3/projects/2299f136823d4a0b81e5279ab66b15b5` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:13:15.609350 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2075c4b-af6d-406b-ab23-3f9aadf06084 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2299f136823d4a0b81e5279ab66b15b5)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:13:15.611250 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2075c4b-af6d-406b-ab23-3f9aadf06084 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:13:15.726254 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-d2075c4b-af6d-406b-ab23-3f9aadf06084 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2299f136823d4a0b81e5279ab66b15b5', 'request_id': 'req-d2075c4b-af6d-406b-ab23-3f9aadf06084'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:13:15.732254 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-d2075c4b-af6d-406b-ab23-3f9aadf06084 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2299f136823d4a0b81e5279ab66b15b5', 'request_id': 'req-d2075c4b-af6d-406b-ab23-3f9aadf06084'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:13:15.737736 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-d2075c4b-af6d-406b-ab23-3f9aadf06084 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2299f136823d4a0b81e5279ab66b15b5', 'request_id': 'req-d2075c4b-af6d-406b-ab23-3f9aadf06084'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:13:15.816452 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-d2075c4b-af6d-406b-ab23-3f9aadf06084 admin admin] The token cache is being invalidate because project 2299f136823d4a0b81e5279ab66b15b5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81602) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:13:15.816742 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-d2075c4b-af6d-406b-ab23-3f9aadf06084 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:13:15.817369 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-d2075c4b-af6d-406b-ab23-3f9aadf06084 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:13:15.818859 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1456/2888] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 24 10:13:15 2022] DELETE /identity/v3/projects/2299f136823d4a0b81e5279ab66b15b5 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:13:15.823652 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69fb7db-b576-4a8d-a865-cfa41c290529 service neutron] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:13:15.833419 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69fb7db-b576-4a8d-a865-cfa41c290529 service neutron] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:13:15.889940 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d111d4-b15b-4cc5-b07c-26f445254e36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:13:15.890571 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d111d4-b15b-4cc5-b07c-26f445254e36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:13:15.890855 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d111d4-b15b-4cc5-b07c-26f445254e36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:13:15.891134 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d111d4-b15b-4cc5-b07c-26f445254e36 admin admin] PATH_INFO: `/v3/users/a83ffb4ee13743a6b9639297a7631bed` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:13:15.892270 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d111d4-b15b-4cc5-b07c-26f445254e36 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a83ffb4ee13743a6b9639297a7631bed)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:13:15.894188 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d111d4-b15b-4cc5-b07c-26f445254e36 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:13:16.096439 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-27d111d4-b15b-4cc5-b07c-26f445254e36 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a83ffb4ee13743a6b9639297a7631bed'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:13:16.097086 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-27d111d4-b15b-4cc5-b07c-26f445254e36 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a83ffb4ee13743a6b9639297a7631bed'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:13:16.109122 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-27d111d4-b15b-4cc5-b07c-26f445254e36 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a83ffb4ee13743a6b9639297a7631bed'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:13:16.113619 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:13:16.116472 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-27d111d4-b15b-4cc5-b07c-26f445254e36 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a83ffb4ee13743a6b9639297a7631bed'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:13:16.186135 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-27d111d4-b15b-4cc5-b07c-26f445254e36 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a83ffb4ee13743a6b9639297a7631bed'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:13:16.192206 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1433/2889] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 24 10:13:15 2022] DELETE /identity/v3/users/a83ffb4ee13743a6b9639297a7631bed => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:13:16.593974 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2075c4b-af6d-406b-ab23-3f9aadf06084 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:13:16.595860 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-d2075c4b-af6d-406b-ab23-3f9aadf06084 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:13:16.612153 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2075c4b-af6d-406b-ab23-3f9aadf06084 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:13:16.624504 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:13:16.668763 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28e82e6-477e-460d-aebb-eb41ac2944dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:13:16.669427 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28e82e6-477e-460d-aebb-eb41ac2944dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:13:16.669739 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28e82e6-477e-460d-aebb-eb41ac2944dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:13:16.670015 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28e82e6-477e-460d-aebb-eb41ac2944dc admin admin] PATH_INFO: `/v3/projects/7ce8b54d7d7c47119af3ed2864882be9` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:13:16.671172 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28e82e6-477e-460d-aebb-eb41ac2944dc admin admin] RBAC: Authorizing `identity:delete_project(project_id=7ce8b54d7d7c47119af3ed2864882be9)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:13:16.673119 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28e82e6-477e-460d-aebb-eb41ac2944dc admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:13:16.796294 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-d28e82e6-477e-460d-aebb-eb41ac2944dc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ce8b54d7d7c47119af3ed2864882be9', 'request_id': 'req-d28e82e6-477e-460d-aebb-eb41ac2944dc'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:13:16.802934 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-d28e82e6-477e-460d-aebb-eb41ac2944dc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ce8b54d7d7c47119af3ed2864882be9', 'request_id': 'req-d28e82e6-477e-460d-aebb-eb41ac2944dc'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:13:16.808676 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-d28e82e6-477e-460d-aebb-eb41ac2944dc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ce8b54d7d7c47119af3ed2864882be9', 'request_id': 'req-d28e82e6-477e-460d-aebb-eb41ac2944dc'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:13:16.874349 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-d28e82e6-477e-460d-aebb-eb41ac2944dc admin admin] The token cache is being invalidate because project 7ce8b54d7d7c47119af3ed2864882be9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81602) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:13:16.874578 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-d28e82e6-477e-460d-aebb-eb41ac2944dc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:13:16.875200 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-d28e82e6-477e-460d-aebb-eb41ac2944dc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:13:16.876726 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1457/2890] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 24 10:13:16 2022] DELETE /identity/v3/projects/7ce8b54d7d7c47119af3ed2864882be9 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:13:16.882036 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d42ba53-1263-4af7-bf3b-f98168b53dab None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:13:16.884380 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d42ba53-1263-4af7-bf3b-f98168b53dab None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:13:16.884380 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d42ba53-1263-4af7-bf3b-f98168b53dab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:13:16.905184 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-0d42ba53-1263-4af7-bf3b-f98168b53dab None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:13:16.906394 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-0d42ba53-1263-4af7-bf3b-f98168b53dab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:13:16.967270 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1434/2891] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 10:13:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:13:16.976304 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc97d3a5-385b-45f6-b50f-bc9800bb3ef2 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:13:16.976587 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc97d3a5-385b-45f6-b50f-bc9800bb3ef2 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:13:16.976885 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc97d3a5-385b-45f6-b50f-bc9800bb3ef2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:13:16.996492 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-bc97d3a5-385b-45f6-b50f-bc9800bb3ef2 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:13:16.997725 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-bc97d3a5-385b-45f6-b50f-bc9800bb3ef2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:13:17.005712 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1458/2892] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 10:13:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:13:17.010438 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d42ba53-1263-4af7-bf3b-f98168b53dab None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:13:17.019961 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d42ba53-1263-4af7-bf3b-f98168b53dab None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:13:17.025162 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24dd1a6-4297-4577-9711-8cd0af75536c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:13:17.025756 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24dd1a6-4297-4577-9711-8cd0af75536c admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:13:17.026034 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24dd1a6-4297-4577-9711-8cd0af75536c admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:13:17.026328 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24dd1a6-4297-4577-9711-8cd0af75536c admin admin] PATH_INFO: `/v3/domains` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:13:17.026971 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24dd1a6-4297-4577-9711-8cd0af75536c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:13:17.027275 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24dd1a6-4297-4577-9711-8cd0af75536c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:13:17.028898 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24dd1a6-4297-4577-9711-8cd0af75536c admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:13:17.040393 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1435/2893] 10.222.0.16 () {58 vars in 1153 bytes} [Wed Aug 24 10:13:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:13:17.044699 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc97d3a5-385b-45f6-b50f-bc9800bb3ef2 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:13:17.053936 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc97d3a5-385b-45f6-b50f-bc9800bb3ef2 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:13:17.058506 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed018b5-1204-40ce-b82e-3ea3061e0692 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:13:17.059090 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed018b5-1204-40ce-b82e-3ea3061e0692 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:13:17.059365 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed018b5-1204-40ce-b82e-3ea3061e0692 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:13:17.059712 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed018b5-1204-40ce-b82e-3ea3061e0692 admin admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:13:17.060330 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed018b5-1204-40ce-b82e-3ea3061e0692 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:13:17.062202 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed018b5-1204-40ce-b82e-3ea3061e0692 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:13:17.147440 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1459/2894] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 24 10:13:17 2022] POST /identity/v3/projects => generated 373 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:13:17.152137 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24dd1a6-4297-4577-9711-8cd0af75536c admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:13:17.161954 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24dd1a6-4297-4577-9711-8cd0af75536c admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:13:17.217544 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f74c4b8-2409-4170-8bb2-deb37ed49c41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:13:17.218137 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f74c4b8-2409-4170-8bb2-deb37ed49c41 admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:13:17.218414 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f74c4b8-2409-4170-8bb2-deb37ed49c41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:13:17.218680 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f74c4b8-2409-4170-8bb2-deb37ed49c41 admin admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:13:17.219291 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f74c4b8-2409-4170-8bb2-deb37ed49c41 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:13:17.221508 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f74c4b8-2409-4170-8bb2-deb37ed49c41 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:13:17.320324 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1436/2895] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 24 10:13:17 2022] POST /identity/v3/users => generated 287 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:13:17.325181 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed018b5-1204-40ce-b82e-3ea3061e0692 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:13:17.335542 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed018b5-1204-40ce-b82e-3ea3061e0692 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:13:17.340286 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde262fe-1a6d-45e4-b8bd-f5c75722d552 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:13:17.340882 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde262fe-1a6d-45e4-b8bd-f5c75722d552 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:13:17.341160 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde262fe-1a6d-45e4-b8bd-f5c75722d552 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:13:17.341428 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde262fe-1a6d-45e4-b8bd-f5c75722d552 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:13:17.342060 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde262fe-1a6d-45e4-b8bd-f5c75722d552 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:13:17.343702 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde262fe-1a6d-45e4-b8bd-f5c75722d552 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:13:17.353733 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1460/2896] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 24 10:13:17 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:13:17.357681 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f74c4b8-2409-4170-8bb2-deb37ed49c41 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:13:17.367750 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f74c4b8-2409-4170-8bb2-deb37ed49c41 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:13:17.372824 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86f3174-98ca-4cb8-9c8c-af14fef21416 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:13:17.373403 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86f3174-98ca-4cb8-9c8c-af14fef21416 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:13:17.373681 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86f3174-98ca-4cb8-9c8c-af14fef21416 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:13:17.373952 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86f3174-98ca-4cb8-9c8c-af14fef21416 admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:13:17.374592 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86f3174-98ca-4cb8-9c8c-af14fef21416 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:13:17.376209 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86f3174-98ca-4cb8-9c8c-af14fef21416 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:13:17.387405 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1437/2897] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 24 10:13:17 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:13:17.391275 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde262fe-1a6d-45e4-b8bd-f5c75722d552 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:13:17.401558 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde262fe-1a6d-45e4-b8bd-f5c75722d552 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:13:17.405980 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f0947f-fdff-4d61-b8a5-d60a75f6bfeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:13:17.406613 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f0947f-fdff-4d61-b8a5-d60a75f6bfeb admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:13:17.406889 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f0947f-fdff-4d61-b8a5-d60a75f6bfeb admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:13:17.407158 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f0947f-fdff-4d61-b8a5-d60a75f6bfeb admin admin] PATH_INFO: `/v3/projects/0e88ea812ae24eabb6c4309760daacce/users/53075cd97b544fe4a95e199f5abb250a/roles/136cdf78e00a426997dd4e37faff489b` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:13:17.421620 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f0947f-fdff-4d61-b8a5-d60a75f6bfeb admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e88ea812ae24eabb6c4309760daacce, user_id=53075cd97b544fe4a95e199f5abb250a, role_id=136cdf78e00a426997dd4e37faff489b)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:13:17.424268 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f0947f-fdff-4d61-b8a5-d60a75f6bfeb admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:13:17.511513 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1461/2898] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 24 10:13:17 2022] PUT /identity/v3/projects/0e88ea812ae24eabb6c4309760daacce/users/53075cd97b544fe4a95e199f5abb250a/roles/136cdf78e00a426997dd4e37faff489b => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 10:13:17.521635 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbce2487-cd15-4066-b445-d7161cab130e None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:13:17.521916 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbce2487-cd15-4066-b445-d7161cab130e None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:13:17.522178 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbce2487-cd15-4066-b445-d7161cab130e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:13:17.550120 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-dbce2487-cd15-4066-b445-d7161cab130e None None] MFA Rules not processed for user `53075cd97b544fe4a95e199f5abb250a`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:13:17.551316 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-dbce2487-cd15-4066-b445-d7161cab130e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:13:17.590894 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:13:17.613159 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1438/2899] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 10:13:17 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:13:17.624208 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f0947f-fdff-4d61-b8a5-d60a75f6bfeb admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:13:17.625953 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-e1f0947f-fdff-4d61-b8a5-d60a75f6bfeb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:13:17.671461 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:13:17.682392 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f0947f-fdff-4d61-b8a5-d60a75f6bfeb admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:13:17.686012 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a876fb-bb5b-484d-bd4a-be0ba7f35e5f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c32b83577c46488906fe409a790b24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:13:17.686607 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a876fb-bb5b-484d-bd4a-be0ba7f35e5f service cinder] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:13:17.686924 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a876fb-bb5b-484d-bd4a-be0ba7f35e5f service cinder] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:13:17.687195 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a876fb-bb5b-484d-bd4a-be0ba7f35e5f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:13:17.695259 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a876fb-bb5b-484d-bd4a-be0ba7f35e5f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:13:17.697306 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a876fb-bb5b-484d-bd4a-be0ba7f35e5f service cinder] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:13:17.706440 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1462/2900] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 10:13:17 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:13:21.604108 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbce2487-cd15-4066-b445-d7161cab130e None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:13:21.664662 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbce2487-cd15-4066-b445-d7161cab130e None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:13:21.669704 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa16c89-870e-452c-b823-ffce07e70ed1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:13:21.670350 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa16c89-870e-452c-b823-ffce07e70ed1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:13:21.670628 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa16c89-870e-452c-b823-ffce07e70ed1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:13:21.670923 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa16c89-870e-452c-b823-ffce07e70ed1 admin admin] PATH_INFO: `/v3/users/53075cd97b544fe4a95e199f5abb250a` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:13:21.672102 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa16c89-870e-452c-b823-ffce07e70ed1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=53075cd97b544fe4a95e199f5abb250a)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:13:21.674035 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa16c89-870e-452c-b823-ffce07e70ed1 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:13:21.857758 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-2aa16c89-870e-452c-b823-ffce07e70ed1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '53075cd97b544fe4a95e199f5abb250a'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:13:21.858384 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-2aa16c89-870e-452c-b823-ffce07e70ed1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '53075cd97b544fe4a95e199f5abb250a'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:13:21.870667 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-2aa16c89-870e-452c-b823-ffce07e70ed1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '53075cd97b544fe4a95e199f5abb250a'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:13:21.877406 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-2aa16c89-870e-452c-b823-ffce07e70ed1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '53075cd97b544fe4a95e199f5abb250a'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:13:21.952016 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-2aa16c89-870e-452c-b823-ffce07e70ed1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '53075cd97b544fe4a95e199f5abb250a'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:13:21.957974 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1439/2901] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 24 10:13:21 2022] DELETE /identity/v3/users/53075cd97b544fe4a95e199f5abb250a => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:13:21.963385 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855527f0-fe1b-49c2-86a4-58e0ee9a7741 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:13:21.963919 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855527f0-fe1b-49c2-86a4-58e0ee9a7741 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:13:21.964023 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855527f0-fe1b-49c2-86a4-58e0ee9a7741 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:13:21.983358 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-855527f0-fe1b-49c2-86a4-58e0ee9a7741 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:13:21.984670 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-855527f0-fe1b-49c2-86a4-58e0ee9a7741 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:13:22.043959 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1463/2902] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 10:13:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:13:22.056825 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa16c89-870e-452c-b823-ffce07e70ed1 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:13:22.058238 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-2aa16c89-870e-452c-b823-ffce07e70ed1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:13:22.117410 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa16c89-870e-452c-b823-ffce07e70ed1 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:13:22.120914 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f849593e-dd1e-4154-bef7-0c6e9425a9db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e5260e798404e8bb12b25ca8ea97052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:13:22.121516 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f849593e-dd1e-4154-bef7-0c6e9425a9db service neutron] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:13:22.121796 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f849593e-dd1e-4154-bef7-0c6e9425a9db service neutron] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:13:22.122084 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f849593e-dd1e-4154-bef7-0c6e9425a9db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:13:22.130887 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f849593e-dd1e-4154-bef7-0c6e9425a9db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:13:22.132351 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f849593e-dd1e-4154-bef7-0c6e9425a9db service neutron] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:13:22.142437 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1440/2903] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 10:13:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:13:22.548074 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855527f0-fe1b-49c2-86a4-58e0ee9a7741 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:13:22.549497 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-855527f0-fe1b-49c2-86a4-58e0ee9a7741 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:13:22.563141 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855527f0-fe1b-49c2-86a4-58e0ee9a7741 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:13:22.568017 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f44cda-27f1-45f0-b5c7-9ac2f8ff44bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:13:22.570006 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f44cda-27f1-45f0-b5c7-9ac2f8ff44bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:13:22.570953 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f44cda-27f1-45f0-b5c7-9ac2f8ff44bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:13:22.571244 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f44cda-27f1-45f0-b5c7-9ac2f8ff44bd admin admin] PATH_INFO: `/v3/projects/0e88ea812ae24eabb6c4309760daacce` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:13:22.573204 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f44cda-27f1-45f0-b5c7-9ac2f8ff44bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=0e88ea812ae24eabb6c4309760daacce)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:13:22.575162 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f44cda-27f1-45f0-b5c7-9ac2f8ff44bd admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:13:22.679765 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-41f44cda-27f1-45f0-b5c7-9ac2f8ff44bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0e88ea812ae24eabb6c4309760daacce', 'request_id': 'req-41f44cda-27f1-45f0-b5c7-9ac2f8ff44bd'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:13:22.686520 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-41f44cda-27f1-45f0-b5c7-9ac2f8ff44bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0e88ea812ae24eabb6c4309760daacce', 'request_id': 'req-41f44cda-27f1-45f0-b5c7-9ac2f8ff44bd'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:13:22.692050 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-41f44cda-27f1-45f0-b5c7-9ac2f8ff44bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0e88ea812ae24eabb6c4309760daacce', 'request_id': 'req-41f44cda-27f1-45f0-b5c7-9ac2f8ff44bd'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:13:22.780463 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-41f44cda-27f1-45f0-b5c7-9ac2f8ff44bd admin admin] The token cache is being invalidate because project 0e88ea812ae24eabb6c4309760daacce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81602) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:13:22.780755 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-41f44cda-27f1-45f0-b5c7-9ac2f8ff44bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:13:22.781398 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-41f44cda-27f1-45f0-b5c7-9ac2f8ff44bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:13:22.782874 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1464/2904] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 24 10:13:22 2022] DELETE /identity/v3/projects/0e88ea812ae24eabb6c4309760daacce => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:13:22.788016 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8805b880-ae9f-41ec-973c-a7b42eac44eb None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:13:22.788481 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8805b880-ae9f-41ec-973c-a7b42eac44eb None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:13:22.788586 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8805b880-ae9f-41ec-973c-a7b42eac44eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:13:22.796967 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:13:22.808247 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-8805b880-ae9f-41ec-973c-a7b42eac44eb None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:13:22.809440 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-8805b880-ae9f-41ec-973c-a7b42eac44eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:13:22.867700 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1441/2905] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 10:13:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:13:22.876027 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e75d8b-1e49-4c90-9eaf-c12a05dc1d6d None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:13:22.876317 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e75d8b-1e49-4c90-9eaf-c12a05dc1d6d None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:13:22.878337 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e75d8b-1e49-4c90-9eaf-c12a05dc1d6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:13:22.896146 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-43e75d8b-1e49-4c90-9eaf-c12a05dc1d6d None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:13:22.897353 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-43e75d8b-1e49-4c90-9eaf-c12a05dc1d6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:13:22.905592 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1465/2906] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 10:13:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:13:22.909906 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8805b880-ae9f-41ec-973c-a7b42eac44eb None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:13:22.920225 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8805b880-ae9f-41ec-973c-a7b42eac44eb None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:13:22.924888 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d3feae-69cd-4095-980b-28eb382aabae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:13:22.929981 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d3feae-69cd-4095-980b-28eb382aabae admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:13:22.929981 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d3feae-69cd-4095-980b-28eb382aabae admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:13:22.929981 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d3feae-69cd-4095-980b-28eb382aabae admin admin] PATH_INFO: `/v3/domains` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:13:22.929981 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d3feae-69cd-4095-980b-28eb382aabae admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:13:22.929981 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d3feae-69cd-4095-980b-28eb382aabae admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:13:22.929981 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d3feae-69cd-4095-980b-28eb382aabae admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:13:22.941156 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1442/2907] 10.222.0.16 () {58 vars in 1153 bytes} [Wed Aug 24 10:13:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:13:22.945462 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e75d8b-1e49-4c90-9eaf-c12a05dc1d6d None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:13:22.954831 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e75d8b-1e49-4c90-9eaf-c12a05dc1d6d None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:13:22.959506 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9d1af7-41c0-4ede-b94a-f5c8ecbcf437 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:13:22.960130 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9d1af7-41c0-4ede-b94a-f5c8ecbcf437 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:13:22.960410 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9d1af7-41c0-4ede-b94a-f5c8ecbcf437 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:13:22.960681 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9d1af7-41c0-4ede-b94a-f5c8ecbcf437 admin admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:13:22.961301 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9d1af7-41c0-4ede-b94a-f5c8ecbcf437 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:13:22.963184 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9d1af7-41c0-4ede-b94a-f5c8ecbcf437 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:13:23.053524 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1466/2908] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 24 10:13:22 2022] POST /identity/v3/projects => generated 369 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:13:23.058266 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d3feae-69cd-4095-980b-28eb382aabae admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:13:23.068375 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d3feae-69cd-4095-980b-28eb382aabae admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:13:23.124969 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea3cddd-4815-464b-8143-5346df7bd862 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:13:23.125564 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea3cddd-4815-464b-8143-5346df7bd862 admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:13:23.125844 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea3cddd-4815-464b-8143-5346df7bd862 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:13:23.126145 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea3cddd-4815-464b-8143-5346df7bd862 admin admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:13:23.126760 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea3cddd-4815-464b-8143-5346df7bd862 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:13:23.128697 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea3cddd-4815-464b-8143-5346df7bd862 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:13:23.211356 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1443/2909] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 24 10:13:23 2022] POST /identity/v3/users => generated 285 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:13:23.215729 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9d1af7-41c0-4ede-b94a-f5c8ecbcf437 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:13:23.226308 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9d1af7-41c0-4ede-b94a-f5c8ecbcf437 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:13:23.231272 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b34b65-21fc-4394-b976-c5644712dce5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:13:23.231897 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b34b65-21fc-4394-b976-c5644712dce5 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:13:23.232176 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b34b65-21fc-4394-b976-c5644712dce5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:13:23.232442 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b34b65-21fc-4394-b976-c5644712dce5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:13:23.233074 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b34b65-21fc-4394-b976-c5644712dce5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:13:23.234635 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b34b65-21fc-4394-b976-c5644712dce5 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:13:23.245651 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1467/2910] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 24 10:13:23 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:13:23.249614 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea3cddd-4815-464b-8143-5346df7bd862 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:13:23.259627 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea3cddd-4815-464b-8143-5346df7bd862 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:13:23.264232 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84143cd-62b9-49db-b03d-aeb92fa90919 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:13:23.270929 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84143cd-62b9-49db-b03d-aeb92fa90919 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:13:23.270929 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84143cd-62b9-49db-b03d-aeb92fa90919 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:13:23.270929 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84143cd-62b9-49db-b03d-aeb92fa90919 admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:13:23.270929 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84143cd-62b9-49db-b03d-aeb92fa90919 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:13:23.270929 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84143cd-62b9-49db-b03d-aeb92fa90919 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:13:23.278372 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1444/2911] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 24 10:13:23 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:13:23.282293 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b34b65-21fc-4394-b976-c5644712dce5 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:13:23.292835 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b34b65-21fc-4394-b976-c5644712dce5 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:13:23.297443 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98443c35-bade-4978-8a73-76fa71c19745 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:13:23.298090 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98443c35-bade-4978-8a73-76fa71c19745 admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:13:23.298393 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98443c35-bade-4978-8a73-76fa71c19745 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:13:23.298694 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98443c35-bade-4978-8a73-76fa71c19745 admin admin] PATH_INFO: `/v3/projects/a71c67ec7f0740b8b50a5ed9f52cecd2/users/f63dc6a04f004e85af43e70e26b2b74c/roles/136cdf78e00a426997dd4e37faff489b` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:13:23.313643 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98443c35-bade-4978-8a73-76fa71c19745 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a71c67ec7f0740b8b50a5ed9f52cecd2, user_id=f63dc6a04f004e85af43e70e26b2b74c, role_id=136cdf78e00a426997dd4e37faff489b)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:13:23.316322 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98443c35-bade-4978-8a73-76fa71c19745 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:13:23.423432 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1468/2912] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 24 10:13:23 2022] PUT /identity/v3/projects/a71c67ec7f0740b8b50a5ed9f52cecd2/users/f63dc6a04f004e85af43e70e26b2b74c/roles/136cdf78e00a426997dd4e37faff489b => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 10:13:23.434387 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f35103a-13f4-44d0-93be-525f31c01568 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:13:23.439013 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f35103a-13f4-44d0-93be-525f31c01568 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:13:23.439013 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f35103a-13f4-44d0-93be-525f31c01568 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:13:23.462090 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-8f35103a-13f4-44d0-93be-525f31c01568 None None] MFA Rules not processed for user `f63dc6a04f004e85af43e70e26b2b74c`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:13:23.463816 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-8f35103a-13f4-44d0-93be-525f31c01568 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:13:23.524870 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:13:23.528231 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1445/2913] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 10:13:23 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:13:23.540156 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98443c35-bade-4978-8a73-76fa71c19745 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:13:23.542549 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-98443c35-bade-4978-8a73-76fa71c19745 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:13:23.547226 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:13:23.619008 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98443c35-bade-4978-8a73-76fa71c19745 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:13:23.622570 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f29cce6-25d6-4498-bb02-81f6bb242cf3 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea2666d430a646d386104addf1f42888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:13:23.623167 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f29cce6-25d6-4498-bb02-81f6bb242cf3 service glance] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:13:23.623443 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f29cce6-25d6-4498-bb02-81f6bb242cf3 service glance] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:13:23.623768 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f29cce6-25d6-4498-bb02-81f6bb242cf3 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:13:23.632277 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f29cce6-25d6-4498-bb02-81f6bb242cf3 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:13:23.633765 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f29cce6-25d6-4498-bb02-81f6bb242cf3 service glance] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:13:23.644207 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1469/2914] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 10:13:23 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:13:33.782989 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=33 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:14:15.034440 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f35103a-13f4-44d0-93be-525f31c01568 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:14:15.096911 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f35103a-13f4-44d0-93be-525f31c01568 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:14:15.101338 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77ee823-2f1a-4226-83a4-ac89b0dc099c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:14:15.101976 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77ee823-2f1a-4226-83a4-ac89b0dc099c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:14:15.102263 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77ee823-2f1a-4226-83a4-ac89b0dc099c admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:14:15.102539 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77ee823-2f1a-4226-83a4-ac89b0dc099c admin admin] PATH_INFO: `/v3/users/f63dc6a04f004e85af43e70e26b2b74c` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:14:15.103710 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77ee823-2f1a-4226-83a4-ac89b0dc099c admin admin] RBAC: Authorizing `identity:delete_user(user_id=f63dc6a04f004e85af43e70e26b2b74c)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:14:15.105636 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77ee823-2f1a-4226-83a4-ac89b0dc099c admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:14:15.264686 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-b77ee823-2f1a-4226-83a4-ac89b0dc099c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f63dc6a04f004e85af43e70e26b2b74c'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:14:15.265367 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-b77ee823-2f1a-4226-83a4-ac89b0dc099c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f63dc6a04f004e85af43e70e26b2b74c'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:14:15.277447 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-b77ee823-2f1a-4226-83a4-ac89b0dc099c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f63dc6a04f004e85af43e70e26b2b74c'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:14:15.284331 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-b77ee823-2f1a-4226-83a4-ac89b0dc099c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f63dc6a04f004e85af43e70e26b2b74c'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:14:15.363559 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-b77ee823-2f1a-4226-83a4-ac89b0dc099c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f63dc6a04f004e85af43e70e26b2b74c'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:14:15.370688 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1446/2915] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 24 10:14:15 2022] DELETE /identity/v3/users/f63dc6a04f004e85af43e70e26b2b74c => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:14:15.375329 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be393673-24e2-4826-b064-e1babe21cdaf None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:14:15.375860 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be393673-24e2-4826-b064-e1babe21cdaf None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:14:15.375957 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be393673-24e2-4826-b064-e1babe21cdaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:14:15.403786 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-be393673-24e2-4826-b064-e1babe21cdaf None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:14:15.404986 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-be393673-24e2-4826-b064-e1babe21cdaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:14:15.462598 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1470/2916] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 10:14:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:14:15.472454 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77ee823-2f1a-4226-83a4-ac89b0dc099c admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:14:15.473865 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-b77ee823-2f1a-4226-83a4-ac89b0dc099c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:14:15.530154 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77ee823-2f1a-4226-83a4-ac89b0dc099c admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:14:15.533732 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ea62d8-82f4-41f1-be82-71730a1ad5c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e5260e798404e8bb12b25ca8ea97052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:14:15.534331 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ea62d8-82f4-41f1-be82-71730a1ad5c7 service neutron] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:14:15.534608 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ea62d8-82f4-41f1-be82-71730a1ad5c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:14:15.534880 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ea62d8-82f4-41f1-be82-71730a1ad5c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:14:15.544122 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ea62d8-82f4-41f1-be82-71730a1ad5c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:14:15.545556 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ea62d8-82f4-41f1-be82-71730a1ad5c7 service neutron] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:14:15.554688 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1447/2917] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 10:14:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:14:15.948101 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be393673-24e2-4826-b064-e1babe21cdaf None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:14:15.949534 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-be393673-24e2-4826-b064-e1babe21cdaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:14:15.961676 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be393673-24e2-4826-b064-e1babe21cdaf None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:14:15.967014 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aeeb0c0-6dd9-4255-a26f-954b68457196 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:14:15.967679 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aeeb0c0-6dd9-4255-a26f-954b68457196 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:14:15.967963 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aeeb0c0-6dd9-4255-a26f-954b68457196 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:14:15.968235 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aeeb0c0-6dd9-4255-a26f-954b68457196 admin admin] PATH_INFO: `/v3/projects/a71c67ec7f0740b8b50a5ed9f52cecd2` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:14:15.969446 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aeeb0c0-6dd9-4255-a26f-954b68457196 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a71c67ec7f0740b8b50a5ed9f52cecd2)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:14:15.971610 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aeeb0c0-6dd9-4255-a26f-954b68457196 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:14:16.066015 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-6aeeb0c0-6dd9-4255-a26f-954b68457196 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a71c67ec7f0740b8b50a5ed9f52cecd2', 'request_id': 'req-6aeeb0c0-6dd9-4255-a26f-954b68457196'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:14:16.072648 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-6aeeb0c0-6dd9-4255-a26f-954b68457196 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a71c67ec7f0740b8b50a5ed9f52cecd2', 'request_id': 'req-6aeeb0c0-6dd9-4255-a26f-954b68457196'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:14:16.078115 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-6aeeb0c0-6dd9-4255-a26f-954b68457196 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a71c67ec7f0740b8b50a5ed9f52cecd2', 'request_id': 'req-6aeeb0c0-6dd9-4255-a26f-954b68457196'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:14:16.147128 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-6aeeb0c0-6dd9-4255-a26f-954b68457196 admin admin] The token cache is being invalidate because project a71c67ec7f0740b8b50a5ed9f52cecd2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81602) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:14:16.147412 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-6aeeb0c0-6dd9-4255-a26f-954b68457196 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:14:16.148056 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-6aeeb0c0-6dd9-4255-a26f-954b68457196 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:14:16.149594 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1471/2918] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 24 10:14:15 2022] DELETE /identity/v3/projects/a71c67ec7f0740b8b50a5ed9f52cecd2 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 10:14:16.156418 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745842fe-134c-4e6a-a004-bc1ba38753f4 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:14:16.156931 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745842fe-134c-4e6a-a004-bc1ba38753f4 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:14:16.157046 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745842fe-134c-4e6a-a004-bc1ba38753f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:14:16.184542 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:14:16.205098 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-745842fe-134c-4e6a-a004-bc1ba38753f4 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:14:16.206230 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-745842fe-134c-4e6a-a004-bc1ba38753f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:14:16.265187 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1448/2919] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 10:14:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:14:16.273064 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef46e83a-4718-4e78-b5fb-29fa3f960d18 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:14:16.273971 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef46e83a-4718-4e78-b5fb-29fa3f960d18 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:14:16.273971 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef46e83a-4718-4e78-b5fb-29fa3f960d18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:14:16.293360 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-ef46e83a-4718-4e78-b5fb-29fa3f960d18 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:14:16.294554 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-ef46e83a-4718-4e78-b5fb-29fa3f960d18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:14:16.302468 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1472/2920] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 10:14:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:14:16.306642 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745842fe-134c-4e6a-a004-bc1ba38753f4 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:14:16.316178 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745842fe-134c-4e6a-a004-bc1ba38753f4 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:14:16.320680 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfaf7453-af3c-4704-be63-f19554b8298e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:14:16.321280 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaf7453-af3c-4704-be63-f19554b8298e admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:14:16.322074 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaf7453-af3c-4704-be63-f19554b8298e admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:14:16.322346 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaf7453-af3c-4704-be63-f19554b8298e admin admin] PATH_INFO: `/v3/domains` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:14:16.322987 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfaf7453-af3c-4704-be63-f19554b8298e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:14:16.323290 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfaf7453-af3c-4704-be63-f19554b8298e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:14:16.324923 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfaf7453-af3c-4704-be63-f19554b8298e admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:14:16.336353 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1449/2921] 10.222.0.16 () {58 vars in 1153 bytes} [Wed Aug 24 10:14:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:14:16.341172 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef46e83a-4718-4e78-b5fb-29fa3f960d18 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:14:16.350454 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef46e83a-4718-4e78-b5fb-29fa3f960d18 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:14:16.355102 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3cc527-51f7-43f9-ba8e-6b8b0222e3ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:14:16.355724 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3cc527-51f7-43f9-ba8e-6b8b0222e3ad admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:14:16.356033 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3cc527-51f7-43f9-ba8e-6b8b0222e3ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:14:16.356276 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3cc527-51f7-43f9-ba8e-6b8b0222e3ad admin admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:14:16.356904 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3cc527-51f7-43f9-ba8e-6b8b0222e3ad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:14:16.358781 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3cc527-51f7-43f9-ba8e-6b8b0222e3ad admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:14:16.452061 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1473/2922] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 24 10:14:16 2022] POST /identity/v3/projects => generated 377 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:14:16.456295 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfaf7453-af3c-4704-be63-f19554b8298e admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:14:16.466714 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfaf7453-af3c-4704-be63-f19554b8298e admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:14:16.522673 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132001fb-e660-4247-9aaa-974ed0906f76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:14:16.523272 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132001fb-e660-4247-9aaa-974ed0906f76 admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:14:16.523550 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132001fb-e660-4247-9aaa-974ed0906f76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:14:16.523855 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132001fb-e660-4247-9aaa-974ed0906f76 admin admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:14:16.524462 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132001fb-e660-4247-9aaa-974ed0906f76 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:14:16.526354 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132001fb-e660-4247-9aaa-974ed0906f76 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:14:16.608656 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1450/2923] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 24 10:14:16 2022] POST /identity/v3/users => generated 289 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:14:16.612816 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3cc527-51f7-43f9-ba8e-6b8b0222e3ad admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:14:16.623457 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3cc527-51f7-43f9-ba8e-6b8b0222e3ad admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:14:16.628018 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce83ed5b-f624-4713-85d7-e13ea3031e5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:14:16.628609 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce83ed5b-f624-4713-85d7-e13ea3031e5f admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:14:16.628906 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce83ed5b-f624-4713-85d7-e13ea3031e5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:14:16.629166 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce83ed5b-f624-4713-85d7-e13ea3031e5f admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:14:16.629807 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce83ed5b-f624-4713-85d7-e13ea3031e5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:14:16.631387 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce83ed5b-f624-4713-85d7-e13ea3031e5f admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:14:16.641762 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1474/2924] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 24 10:14:16 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 10:14:16.645583 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132001fb-e660-4247-9aaa-974ed0906f76 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:14:16.655225 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132001fb-e660-4247-9aaa-974ed0906f76 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:14:16.659808 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33396514-920f-46ec-8e47-663df44b8115 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:14:16.660399 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33396514-920f-46ec-8e47-663df44b8115 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:14:16.660680 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33396514-920f-46ec-8e47-663df44b8115 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:14:16.660953 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33396514-920f-46ec-8e47-663df44b8115 admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:14:16.661594 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33396514-920f-46ec-8e47-663df44b8115 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:14:16.663174 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33396514-920f-46ec-8e47-663df44b8115 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:14:16.673462 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1451/2925] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 24 10:14:16 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:14:16.677334 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce83ed5b-f624-4713-85d7-e13ea3031e5f admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:14:16.687759 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce83ed5b-f624-4713-85d7-e13ea3031e5f admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:14:16.692240 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5e7509-1b8b-4ea1-9e9b-f0d350077e54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:14:16.692871 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5e7509-1b8b-4ea1-9e9b-f0d350077e54 admin admin] REQUEST_METHOD: `PUT` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:14:16.693148 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5e7509-1b8b-4ea1-9e9b-f0d350077e54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:14:16.693417 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5e7509-1b8b-4ea1-9e9b-f0d350077e54 admin admin] PATH_INFO: `/v3/projects/e8f7d973a28745d0a1109ce56927b450/users/89351fa3299b41c5b543f0ccdf9f7527/roles/136cdf78e00a426997dd4e37faff489b` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:14:16.708257 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5e7509-1b8b-4ea1-9e9b-f0d350077e54 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e8f7d973a28745d0a1109ce56927b450, user_id=89351fa3299b41c5b543f0ccdf9f7527, role_id=136cdf78e00a426997dd4e37faff489b)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:14:16.710860 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5e7509-1b8b-4ea1-9e9b-f0d350077e54 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:14:16.791077 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1475/2926] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 24 10:14:16 2022] PUT /identity/v3/projects/e8f7d973a28745d0a1109ce56927b450/users/89351fa3299b41c5b543f0ccdf9f7527/roles/136cdf78e00a426997dd4e37faff489b => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:14:16.796096 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2d04ca-3206-4c13-93d7-610456f66238 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:14:16.796387 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2d04ca-3206-4c13-93d7-610456f66238 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:14:16.796664 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2d04ca-3206-4c13-93d7-610456f66238 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:14:16.816568 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-3c2d04ca-3206-4c13-93d7-610456f66238 None None] MFA Rules not processed for user `d1828412cb04465dbfa15b26582aa6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:14:16.817779 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-3c2d04ca-3206-4c13-93d7-610456f66238 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:14:16.876699 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1452/2927] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 10:14:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:14:16.886322 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5e7509-1b8b-4ea1-9e9b-f0d350077e54 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:14:16.888103 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-dd5e7509-1b8b-4ea1-9e9b-f0d350077e54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:14:16.894600 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:14:16.946864 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5e7509-1b8b-4ea1-9e9b-f0d350077e54 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:14:16.950726 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67792ec0-9207-46eb-8058-14087a8efb50 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e5260e798404e8bb12b25ca8ea97052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:14:16.951338 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67792ec0-9207-46eb-8058-14087a8efb50 service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:14:16.951642 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67792ec0-9207-46eb-8058-14087a8efb50 service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:14:16.951919 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67792ec0-9207-46eb-8058-14087a8efb50 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:14:16.961318 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67792ec0-9207-46eb-8058-14087a8efb50 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:14:16.963408 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67792ec0-9207-46eb-8058-14087a8efb50 service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:14:16.973889 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1476/2928] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 10:14:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:14:22.223228 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8345831-3860-42c5-94dc-d70a1db51221 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:14:22.223838 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8345831-3860-42c5-94dc-d70a1db51221 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:14:22.223838 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8345831-3860-42c5-94dc-d70a1db51221 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:14:22.232694 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:14:22.253405 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-a8345831-3860-42c5-94dc-d70a1db51221 None None] MFA Rules not processed for user `89351fa3299b41c5b543f0ccdf9f7527`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:14:22.254628 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-a8345831-3860-42c5-94dc-d70a1db51221 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:14:22.323741 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1453/2929] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 10:14:22 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:14:22.328247 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67792ec0-9207-46eb-8058-14087a8efb50 service neutron] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:14:22.339073 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67792ec0-9207-46eb-8058-14087a8efb50 service neutron] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:14:22.343880 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a4b7f9-16df-41c0-85c4-1c06a3b5a8e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:14:22.344483 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a4b7f9-16df-41c0-85c4-1c06a3b5a8e9 admin admin] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:14:22.344764 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a4b7f9-16df-41c0-85c4-1c06a3b5a8e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:14:22.345066 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a4b7f9-16df-41c0-85c4-1c06a3b5a8e9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:14:22.345705 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a4b7f9-16df-41c0-85c4-1c06a3b5a8e9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:14:22.347910 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a4b7f9-16df-41c0-85c4-1c06a3b5a8e9 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:14:22.437176 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1477/2930] 10.222.0.16 () {60 vars in 1153 bytes} [Wed Aug 24 10:14:22 2022] POST /identity/v3/projects => generated 377 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:14:22.438413 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8345831-3860-42c5-94dc-d70a1db51221 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:14:22.449955 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8345831-3860-42c5-94dc-d70a1db51221 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:14:22.509930 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a03bb7d-b3d4-4942-b91d-f20c2dc449c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:14:22.510632 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a03bb7d-b3d4-4942-b91d-f20c2dc449c5 admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:14:22.511050 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a03bb7d-b3d4-4942-b91d-f20c2dc449c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:14:22.511291 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a03bb7d-b3d4-4942-b91d-f20c2dc449c5 admin admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:14:22.511982 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a03bb7d-b3d4-4942-b91d-f20c2dc449c5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:14:22.513895 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a03bb7d-b3d4-4942-b91d-f20c2dc449c5 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:14:22.616967 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1454/2931] 10.222.0.16 () {60 vars in 1144 bytes} [Wed Aug 24 10:14:22 2022] POST /identity/v3/users => generated 295 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:14:22.622559 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a4b7f9-16df-41c0-85c4-1c06a3b5a8e9 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:14:22.633163 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a4b7f9-16df-41c0-85c4-1c06a3b5a8e9 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:14:22.638278 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623c019d-864b-441d-94fa-2cc3955236b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:14:22.638877 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623c019d-864b-441d-94fa-2cc3955236b1 admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:14:22.639159 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623c019d-864b-441d-94fa-2cc3955236b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:14:22.639437 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623c019d-864b-441d-94fa-2cc3955236b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:14:22.640252 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623c019d-864b-441d-94fa-2cc3955236b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:14:22.641847 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623c019d-864b-441d-94fa-2cc3955236b1 admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:14:22.652533 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1478/2932] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 24 10:14:22 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:14:22.656622 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a03bb7d-b3d4-4942-b91d-f20c2dc449c5 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:14:22.666632 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a03bb7d-b3d4-4942-b91d-f20c2dc449c5 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:14:22.671103 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c78a437-0b4e-4c27-bc16-b30ff0b08970 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:14:22.671783 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c78a437-0b4e-4c27-bc16-b30ff0b08970 admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:14:22.672066 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c78a437-0b4e-4c27-bc16-b30ff0b08970 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:14:22.672341 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c78a437-0b4e-4c27-bc16-b30ff0b08970 admin admin] PATH_INFO: `/v3/projects/922ea74720354bc488557a97ec9fe087/users/13994c0a59ef4d9ca4956b06a9f07099/roles/15f634d4a2d247ddab90c458e5e1c141` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:14:22.687820 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c78a437-0b4e-4c27-bc16-b30ff0b08970 admin admin] RBAC: Authorizing `identity:create_grant(project_id=922ea74720354bc488557a97ec9fe087, user_id=13994c0a59ef4d9ca4956b06a9f07099, role_id=15f634d4a2d247ddab90c458e5e1c141)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:14:22.690826 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c78a437-0b4e-4c27-bc16-b30ff0b08970 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:14:22.761289 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1455/2933] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 24 10:14:22 2022] PUT /identity/v3/projects/922ea74720354bc488557a97ec9fe087/users/13994c0a59ef4d9ca4956b06a9f07099/roles/15f634d4a2d247ddab90c458e5e1c141 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 10:14:22.765447 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623c019d-864b-441d-94fa-2cc3955236b1 admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:14:22.775328 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623c019d-864b-441d-94fa-2cc3955236b1 admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:14:22.831370 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af18daa6-697e-4e66-9c0c-22a3cdd5c8fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:14:22.832030 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af18daa6-697e-4e66-9c0c-22a3cdd5c8fe admin admin] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:14:22.832293 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af18daa6-697e-4e66-9c0c-22a3cdd5c8fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:14:22.832568 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af18daa6-697e-4e66-9c0c-22a3cdd5c8fe admin admin] PATH_INFO: `/v3/roles` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:14:22.833211 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af18daa6-697e-4e66-9c0c-22a3cdd5c8fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:14:22.834799 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af18daa6-697e-4e66-9c0c-22a3cdd5c8fe admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:14:22.844370 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1479/2934] 10.222.0.16 () {58 vars in 1122 bytes} [Wed Aug 24 10:14:22 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:14:22.848794 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c78a437-0b4e-4c27-bc16-b30ff0b08970 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:14:22.860822 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c78a437-0b4e-4c27-bc16-b30ff0b08970 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:14:22.864163 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204e0c2e-32dd-40af-87f8-fe65c7fc757f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:14:22.864821 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204e0c2e-32dd-40af-87f8-fe65c7fc757f admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:14:22.865100 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204e0c2e-32dd-40af-87f8-fe65c7fc757f admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:14:22.865375 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204e0c2e-32dd-40af-87f8-fe65c7fc757f admin admin] PATH_INFO: `/v3/projects/922ea74720354bc488557a97ec9fe087/users/13994c0a59ef4d9ca4956b06a9f07099/roles/15f634d4a2d247ddab90c458e5e1c141` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:14:22.867391 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204e0c2e-32dd-40af-87f8-fe65c7fc757f admin admin] RBAC: Authorizing `identity:create_grant(project_id=922ea74720354bc488557a97ec9fe087, user_id=13994c0a59ef4d9ca4956b06a9f07099, role_id=15f634d4a2d247ddab90c458e5e1c141)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:14:22.870725 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204e0c2e-32dd-40af-87f8-fe65c7fc757f admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:14:22.889182 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1456/2935] 10.222.0.16 () {60 vars in 1483 bytes} [Wed Aug 24 10:14:22 2022] PUT /identity/v3/projects/922ea74720354bc488557a97ec9fe087/users/13994c0a59ef4d9ca4956b06a9f07099/roles/15f634d4a2d247ddab90c458e5e1c141 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 10:14:28.227232 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f9292f-1424-42c8-ad0d-8042d158a27c None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:14:28.227742 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f9292f-1424-42c8-ad0d-8042d158a27c None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:14:28.227833 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f9292f-1424-42c8-ad0d-8042d158a27c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:14:28.280896 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-11f9292f-1424-42c8-ad0d-8042d158a27c None None] MFA Rules not processed for user `13994c0a59ef4d9ca4956b06a9f07099`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:14:28.282103 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-11f9292f-1424-42c8-ad0d-8042d158a27c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:14:28.344683 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:14:28.354271 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1480/2936] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 10:14:28 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:14:28.371167 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204e0c2e-32dd-40af-87f8-fe65c7fc757f admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:14:28.374498 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-204e0c2e-32dd-40af-87f8-fe65c7fc757f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:14:28.451512 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:14:28.487079 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204e0c2e-32dd-40af-87f8-fe65c7fc757f admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:14:28.492718 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23fe4a0-4a56-4247-88b0-dd3dc3096559 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f055f7dc51d43bda9b315ca67f11d58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:14:28.493880 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23fe4a0-4a56-4247-88b0-dd3dc3096559 service nova] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:14:28.494161 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23fe4a0-4a56-4247-88b0-dd3dc3096559 service nova] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:14:28.494437 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23fe4a0-4a56-4247-88b0-dd3dc3096559 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:14:28.555022 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23fe4a0-4a56-4247-88b0-dd3dc3096559 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:14:28.557246 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23fe4a0-4a56-4247-88b0-dd3dc3096559 service nova] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:14:28.566892 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1457/2937] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 10:14:28 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:14:38.505627 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=2 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:14:38.622740 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=47 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:15:06.653181 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe169950-8ee4-43b7-86cd-309c4e1e0556 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:15:06.653711 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe169950-8ee4-43b7-86cd-309c4e1e0556 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:15:06.653711 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe169950-8ee4-43b7-86cd-309c4e1e0556 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:15:06.672062 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-fe169950-8ee4-43b7-86cd-309c4e1e0556 None None] MFA Rules not processed for user `89351fa3299b41c5b543f0ccdf9f7527`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:15:06.673749 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-fe169950-8ee4-43b7-86cd-309c4e1e0556 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:15:06.680510 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1481/2938] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 10:15:06 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:15:06.720899 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23fe4a0-4a56-4247-88b0-dd3dc3096559 service nova] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:15:06.725841 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-f23fe4a0-4a56-4247-88b0-dd3dc3096559 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:15:06.779123 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23fe4a0-4a56-4247-88b0-dd3dc3096559 service nova] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:15:06.782683 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c87983-a404-4f9d-a74a-0e3c05300cf2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c32b83577c46488906fe409a790b24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:15:06.783313 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c87983-a404-4f9d-a74a-0e3c05300cf2 service cinder] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:15:06.783609 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c87983-a404-4f9d-a74a-0e3c05300cf2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:15:06.783900 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c87983-a404-4f9d-a74a-0e3c05300cf2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:15:06.785598 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-27c87983-a404-4f9d-a74a-0e3c05300cf2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:15:06.799215 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c87983-a404-4f9d-a74a-0e3c05300cf2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:15:06.800600 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c87983-a404-4f9d-a74a-0e3c05300cf2 service cinder] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:15:06.811432 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1458/2939] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 10:15:06 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:15:07.095274 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe169950-8ee4-43b7-86cd-309c4e1e0556 None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:15:07.096928 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-fe169950-8ee4-43b7-86cd-309c4e1e0556 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:15:07.151315 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe169950-8ee4-43b7-86cd-309c4e1e0556 None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:15:07.154892 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc054a41-e8f2-4966-9212-94bdb6f70720 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e5260e798404e8bb12b25ca8ea97052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:15:07.155483 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc054a41-e8f2-4966-9212-94bdb6f70720 service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:15:07.155791 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc054a41-e8f2-4966-9212-94bdb6f70720 service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:15:07.156063 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc054a41-e8f2-4966-9212-94bdb6f70720 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:15:07.158261 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-fc054a41-e8f2-4966-9212-94bdb6f70720 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:15:07.167357 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc054a41-e8f2-4966-9212-94bdb6f70720 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:15:07.168841 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc054a41-e8f2-4966-9212-94bdb6f70720 service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:15:07.177353 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1482/2940] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 10:15:07 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:15:07.867867 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c87983-a404-4f9d-a74a-0e3c05300cf2 service cinder] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:15:07.869302 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-27c87983-a404-4f9d-a74a-0e3c05300cf2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:15:07.874546 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc054a41-e8f2-4966-9212-94bdb6f70720 service neutron] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:15:07.883898 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc054a41-e8f2-4966-9212-94bdb6f70720 service neutron] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:15:07.887529 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965af831-c1a2-474d-8e85-62ca5e756bf3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e5260e798404e8bb12b25ca8ea97052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:15:07.888165 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965af831-c1a2-474d-8e85-62ca5e756bf3 service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:15:07.888889 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965af831-c1a2-474d-8e85-62ca5e756bf3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:15:07.894093 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965af831-c1a2-474d-8e85-62ca5e756bf3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:15:07.894093 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-965af831-c1a2-474d-8e85-62ca5e756bf3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:15:07.904598 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965af831-c1a2-474d-8e85-62ca5e756bf3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:15:07.906022 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965af831-c1a2-474d-8e85-62ca5e756bf3 service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:15:07.917938 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1483/2941] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 10:15:07 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 10:15:07.942970 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c87983-a404-4f9d-a74a-0e3c05300cf2 service cinder] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:15:07.948358 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceab7f0a-1b78-48b0-9888-263a0be5dd89 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80cada22039d4f11935412ee578acd05', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:15:07.949037 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceab7f0a-1b78-48b0-9888-263a0be5dd89 service placement] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:15:07.949314 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceab7f0a-1b78-48b0-9888-263a0be5dd89 service placement] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:15:07.949588 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceab7f0a-1b78-48b0-9888-263a0be5dd89 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:15:07.951405 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-ceab7f0a-1b78-48b0-9888-263a0be5dd89 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:15:07.963756 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceab7f0a-1b78-48b0-9888-263a0be5dd89 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:15:07.965334 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceab7f0a-1b78-48b0-9888-263a0be5dd89 service placement] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:15:07.976363 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1459/2942] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 10:15:07 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:15:11.130721 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965af831-c1a2-474d-8e85-62ca5e756bf3 service neutron] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:15:11.140160 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965af831-c1a2-474d-8e85-62ca5e756bf3 service neutron] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:15:11.144064 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d37401b-dbde-4f85-b726-f08b25346c54 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e5260e798404e8bb12b25ca8ea97052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:15:11.144670 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d37401b-dbde-4f85-b726-f08b25346c54 service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:15:11.144946 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d37401b-dbde-4f85-b726-f08b25346c54 service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:15:11.145216 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d37401b-dbde-4f85-b726-f08b25346c54 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:15:11.147223 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-7d37401b-dbde-4f85-b726-f08b25346c54 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:15:11.158535 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d37401b-dbde-4f85-b726-f08b25346c54 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:15:11.159987 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d37401b-dbde-4f85-b726-f08b25346c54 service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:15:11.169002 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1484/2943] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 10:15:11 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:15:11.985149 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceab7f0a-1b78-48b0-9888-263a0be5dd89 service placement] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:15:11.997694 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceab7f0a-1b78-48b0-9888-263a0be5dd89 service placement] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:15:12.003366 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d745152-a39d-47f7-8e21-598833404067 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f055f7dc51d43bda9b315ca67f11d58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:15:12.004021 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d745152-a39d-47f7-8e21-598833404067 service nova] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:15:12.004303 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d745152-a39d-47f7-8e21-598833404067 service nova] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:15:12.004577 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d745152-a39d-47f7-8e21-598833404067 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:15:12.006395 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-6d745152-a39d-47f7-8e21-598833404067 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:15:12.019506 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d745152-a39d-47f7-8e21-598833404067 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:15:12.021162 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d745152-a39d-47f7-8e21-598833404067 service nova] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:15:12.034763 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1460/2944] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 10:15:11 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:15:21.165179 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=41 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:15:22.028896 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=64 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:15:24.768468 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d37401b-dbde-4f85-b726-f08b25346c54 service neutron] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:15:24.769948 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-7d37401b-dbde-4f85-b726-f08b25346c54 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:15:24.785666 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d37401b-dbde-4f85-b726-f08b25346c54 service neutron] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:15:24.791270 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f84028-c501-4366-b434-3ecffd9c3ead service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f055f7dc51d43bda9b315ca67f11d58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:15:24.791906 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f84028-c501-4366-b434-3ecffd9c3ead service nova] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:15:24.792184 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f84028-c501-4366-b434-3ecffd9c3ead service nova] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:15:24.792452 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f84028-c501-4366-b434-3ecffd9c3ead service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:15:24.794123 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-66f84028-c501-4366-b434-3ecffd9c3ead service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:15:24.806265 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f84028-c501-4366-b434-3ecffd9c3ead service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:15:24.807887 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f84028-c501-4366-b434-3ecffd9c3ead service nova] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:15:24.818734 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1485/2945] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 10:15:24 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 10:15:27.957001 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d745152-a39d-47f7-8e21-598833404067 service nova] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:15:27.966616 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d745152-a39d-47f7-8e21-598833404067 service nova] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:15:27.970238 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88cfd59-5567-4919-a808-c68bfbd67ea1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e5260e798404e8bb12b25ca8ea97052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:15:27.970834 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88cfd59-5567-4919-a808-c68bfbd67ea1 service neutron] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:15:27.971115 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88cfd59-5567-4919-a808-c68bfbd67ea1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:15:27.971385 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88cfd59-5567-4919-a808-c68bfbd67ea1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:15:27.980509 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88cfd59-5567-4919-a808-c68bfbd67ea1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:15:27.981917 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88cfd59-5567-4919-a808-c68bfbd67ea1 service neutron] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:15:27.991314 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1461/2946] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 10:15:27 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:15:30.047542 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f84028-c501-4366-b434-3ecffd9c3ead service nova] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:15:30.060163 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f84028-c501-4366-b434-3ecffd9c3ead service nova] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:15:30.066592 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe8b538-85de-4c43-9dda-b010693747c4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f055f7dc51d43bda9b315ca67f11d58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:15:30.067197 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe8b538-85de-4c43-9dda-b010693747c4 service nova] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:15:30.067477 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe8b538-85de-4c43-9dda-b010693747c4 service nova] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:15:30.067851 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe8b538-85de-4c43-9dda-b010693747c4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:15:30.069581 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-5fe8b538-85de-4c43-9dda-b010693747c4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:15:30.082503 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe8b538-85de-4c43-9dda-b010693747c4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:15:30.084042 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe8b538-85de-4c43-9dda-b010693747c4 service nova] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:15:30.094901 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1486/2947] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 10:15:30 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:15:37.987048 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=6 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:15:40.090903 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=12 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:17:36.030186 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88cfd59-5567-4919-a808-c68bfbd67ea1 service neutron] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:36.090958 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88cfd59-5567-4919-a808-c68bfbd67ea1 service neutron] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:36.096001 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcfa445-56bd-47f7-ba60-c3f9846c17c5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80cada22039d4f11935412ee578acd05', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:36.096616 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcfa445-56bd-47f7-ba60-c3f9846c17c5 service placement] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:36.096888 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcfa445-56bd-47f7-ba60-c3f9846c17c5 service placement] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:36.097158 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcfa445-56bd-47f7-ba60-c3f9846c17c5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:36.099132 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-6bcfa445-56bd-47f7-ba60-c3f9846c17c5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:36.132147 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcfa445-56bd-47f7-ba60-c3f9846c17c5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:36.134275 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcfa445-56bd-47f7-ba60-c3f9846c17c5 service placement] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:36.144035 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1462/2948] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 10:17:36 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:17:46.139805 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=36 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:18:02.192911 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592dc91d-3a59-4a4f-ae9e-d84ecedf841d None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:02.193382 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592dc91d-3a59-4a4f-ae9e-d84ecedf841d None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:02.193476 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592dc91d-3a59-4a4f-ae9e-d84ecedf841d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:02.212758 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-592dc91d-3a59-4a4f-ae9e-d84ecedf841d None None] MFA Rules not processed for user `89351fa3299b41c5b543f0ccdf9f7527`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:18:02.213933 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-592dc91d-3a59-4a4f-ae9e-d84ecedf841d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:18:02.220679 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1487/2949] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 10:18:02 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:18:02.525803 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cdf73d-5dc3-4ada-b41a-00f29db5f177 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:02.526381 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cdf73d-5dc3-4ada-b41a-00f29db5f177 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:02.526381 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cdf73d-5dc3-4ada-b41a-00f29db5f177 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:02.571939 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-b1cdf73d-5dc3-4ada-b41a-00f29db5f177 None None] MFA Rules not processed for user `8e5260e798404e8bb12b25ca8ea97052`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:18:02.573139 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-b1cdf73d-5dc3-4ada-b41a-00f29db5f177 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:18:02.579453 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1463/2950] 10.222.0.16 () {58 vars in 1062 bytes} [Wed Aug 24 10:18:02 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:18:02.585233 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592dc91d-3a59-4a4f-ae9e-d84ecedf841d None None] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:18:02.595173 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592dc91d-3a59-4a4f-ae9e-d84ecedf841d None None] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:18:02.598949 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe4681a-076b-4b74-b597-2e64f7db295e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e5260e798404e8bb12b25ca8ea97052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:18:02.599542 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe4681a-076b-4b74-b597-2e64f7db295e service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:02.599850 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe4681a-076b-4b74-b597-2e64f7db295e service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:02.600123 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe4681a-076b-4b74-b597-2e64f7db295e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:02.601856 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-efe4681a-076b-4b74-b597-2e64f7db295e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:18:02.611119 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe4681a-076b-4b74-b597-2e64f7db295e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:18:02.612552 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe4681a-076b-4b74-b597-2e64f7db295e service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:18:02.620293 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1488/2951] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 10:18:02 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:18:05.359468 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc1381e-159a-4907-8be1-9448826fbfb3 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:05.359955 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc1381e-159a-4907-8be1-9448826fbfb3 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:05.360053 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc1381e-159a-4907-8be1-9448826fbfb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:05.401549 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-7cc1381e-159a-4907-8be1-9448826fbfb3 None None] MFA Rules not processed for user `80cada22039d4f11935412ee578acd05`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:18:05.402743 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-7cc1381e-159a-4907-8be1-9448826fbfb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:18:05.411107 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1464/2952] 10.222.0.16 () {58 vars in 1018 bytes} [Wed Aug 24 10:18:05 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:18:05.422354 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe4681a-076b-4b74-b597-2e64f7db295e service neutron] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:18:05.434813 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe4681a-076b-4b74-b597-2e64f7db295e service neutron] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:18:05.439837 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9310eff-1411-4229-a17d-67d979fbe7ff service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80cada22039d4f11935412ee578acd05', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:18:05.440419 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9310eff-1411-4229-a17d-67d979fbe7ff service placement] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:05.440728 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9310eff-1411-4229-a17d-67d979fbe7ff service placement] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:05.440999 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9310eff-1411-4229-a17d-67d979fbe7ff service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:05.450109 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9310eff-1411-4229-a17d-67d979fbe7ff service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:18:05.451742 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9310eff-1411-4229-a17d-67d979fbe7ff service placement] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:18:05.461834 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1489/2953] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 10:18:05 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:18:15.399246 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=22 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:18:15.457481 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=15 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:20:43.484312 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb7dc2b-6511-462c-9603-6d4033eaa55a None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:20:43.484752 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb7dc2b-6511-462c-9603-6d4033eaa55a None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:20:43.484879 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb7dc2b-6511-462c-9603-6d4033eaa55a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:20:43.520939 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-4cb7dc2b-6511-462c-9603-6d4033eaa55a None None] MFA Rules not processed for user `05c32b83577c46488906fe409a790b24`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:20:43.522165 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-4cb7dc2b-6511-462c-9603-6d4033eaa55a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:20:43.538478 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1465/2954] 10.222.0.16 () {58 vars in 1061 bytes} [Wed Aug 24 10:20:43 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:20:43.545240 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9310eff-1411-4229-a17d-67d979fbe7ff service placement] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:20:43.554136 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9310eff-1411-4229-a17d-67d979fbe7ff service placement] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:20:43.557637 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa0bbfc-6af0-4287-83d4-2e16fc6e070e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c32b83577c46488906fe409a790b24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:20:43.558221 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa0bbfc-6af0-4287-83d4-2e16fc6e070e service cinder] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:20:43.558496 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa0bbfc-6af0-4287-83d4-2e16fc6e070e service cinder] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:20:43.558763 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa0bbfc-6af0-4287-83d4-2e16fc6e070e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:20:43.566688 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa0bbfc-6af0-4287-83d4-2e16fc6e070e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:20:43.568137 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa0bbfc-6af0-4287-83d4-2e16fc6e070e service cinder] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:20:43.576729 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1490/2955] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 10:20:43 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 32 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:20:48.500507 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a05f463-b13b-424e-aa13-795db3f20dd4 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:20:48.501035 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a05f463-b13b-424e-aa13-795db3f20dd4 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:20:48.501273 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a05f463-b13b-424e-aa13-795db3f20dd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:20:48.521845 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-7a05f463-b13b-424e-aa13-795db3f20dd4 None None] MFA Rules not processed for user `8e5260e798404e8bb12b25ca8ea97052`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:20:48.523051 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-7a05f463-b13b-424e-aa13-795db3f20dd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:20:48.530347 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1466/2956] 10.222.0.16 () {58 vars in 1062 bytes} [Wed Aug 24 10:20:48 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:20:48.536271 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa0bbfc-6af0-4287-83d4-2e16fc6e070e service cinder] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:20:48.545347 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa0bbfc-6af0-4287-83d4-2e16fc6e070e service cinder] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:20:48.549166 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e8adf7-b20c-49fa-b4eb-f4f471972aa9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e5260e798404e8bb12b25ca8ea97052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:20:48.549765 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e8adf7-b20c-49fa-b4eb-f4f471972aa9 service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:20:48.550046 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e8adf7-b20c-49fa-b4eb-f4f471972aa9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:20:48.550318 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e8adf7-b20c-49fa-b4eb-f4f471972aa9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:20:48.558440 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e8adf7-b20c-49fa-b4eb-f4f471972aa9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:20:48.559856 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e8adf7-b20c-49fa-b4eb-f4f471972aa9 service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:20:48.567816 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1491/2957] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 10:20:48 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 32 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:20:57.162155 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a005700-c08c-4f16-99f5-f012d95ffba0 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:20:57.162640 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a005700-c08c-4f16-99f5-f012d95ffba0 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:20:57.162746 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a005700-c08c-4f16-99f5-f012d95ffba0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:20:57.181651 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-7a005700-c08c-4f16-99f5-f012d95ffba0 None None] MFA Rules not processed for user `89351fa3299b41c5b543f0ccdf9f7527`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:20:57.182832 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-7a005700-c08c-4f16-99f5-f012d95ffba0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:20:57.189470 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1467/2958] 10.222.0.16 () {58 vars in 958 bytes} [Wed Aug 24 10:20:57 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:20:57.227978 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e8adf7-b20c-49fa-b4eb-f4f471972aa9 service neutron] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:20:57.237142 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e8adf7-b20c-49fa-b4eb-f4f471972aa9 service neutron] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:20:57.240772 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdad1bd4-3bb4-4923-9d1a-407366ecbc7f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c32b83577c46488906fe409a790b24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:20:57.241359 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdad1bd4-3bb4-4923-9d1a-407366ecbc7f service cinder] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:20:57.241637 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdad1bd4-3bb4-4923-9d1a-407366ecbc7f service cinder] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:20:57.241908 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdad1bd4-3bb4-4923-9d1a-407366ecbc7f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:20:57.252162 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdad1bd4-3bb4-4923-9d1a-407366ecbc7f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:20:57.253518 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdad1bd4-3bb4-4923-9d1a-407366ecbc7f service cinder] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:20:57.264810 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1492/2959] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 10:20:57 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:20:58.313490 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383e799e-01e0-4be2-801f-f8ebe5d289b9 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:20:58.313971 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383e799e-01e0-4be2-801f-f8ebe5d289b9 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:20:58.314082 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383e799e-01e0-4be2-801f-f8ebe5d289b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:20:58.328753 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdad1bd4-3bb4-4923-9d1a-407366ecbc7f service cinder] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:20:58.336419 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-383e799e-01e0-4be2-801f-f8ebe5d289b9 None None] MFA Rules not processed for user `80cada22039d4f11935412ee578acd05`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:20:58.337653 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-383e799e-01e0-4be2-801f-f8ebe5d289b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:20:58.338261 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdad1bd4-3bb4-4923-9d1a-407366ecbc7f service cinder] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:20:58.342425 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c59715-bb38-4435-a4a7-845ce28910ac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e5260e798404e8bb12b25ca8ea97052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:20:58.343141 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c59715-bb38-4435-a4a7-845ce28910ac service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:20:58.343424 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c59715-bb38-4435-a4a7-845ce28910ac service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:20:58.343729 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c59715-bb38-4435-a4a7-845ce28910ac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:20:58.347059 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1468/2960] 10.222.0.16 () {58 vars in 1018 bytes} [Wed Aug 24 10:20:58 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:20:58.357512 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c59715-bb38-4435-a4a7-845ce28910ac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:20:58.359168 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c59715-bb38-4435-a4a7-845ce28910ac service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:20:58.360091 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383e799e-01e0-4be2-801f-f8ebe5d289b9 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:20:58.361434 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-383e799e-01e0-4be2-801f-f8ebe5d289b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:20:58.370529 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1493/2961] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 10:20:58 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:20:58.374216 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383e799e-01e0-4be2-801f-f8ebe5d289b9 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:20:58.379295 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182c78a2-3798-4fb8-9cb6-7966a9976ed9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80cada22039d4f11935412ee578acd05', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:20:58.379929 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182c78a2-3798-4fb8-9cb6-7966a9976ed9 service placement] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:20:58.380216 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182c78a2-3798-4fb8-9cb6-7966a9976ed9 service placement] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:20:58.380487 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182c78a2-3798-4fb8-9cb6-7966a9976ed9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:20:58.391112 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182c78a2-3798-4fb8-9cb6-7966a9976ed9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:20:58.392635 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182c78a2-3798-4fb8-9cb6-7966a9976ed9 service placement] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:20:58.402903 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1469/2962] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 10:20:58 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 10:21:02.048001 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c59715-bb38-4435-a4a7-845ce28910ac service neutron] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:21:02.060817 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c59715-bb38-4435-a4a7-845ce28910ac service neutron] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:21:02.066661 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2feb6655-4b5e-48fb-b115-00d31e5a2f21 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f055f7dc51d43bda9b315ca67f11d58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:21:02.067358 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2feb6655-4b5e-48fb-b115-00d31e5a2f21 service nova] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:21:02.067673 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2feb6655-4b5e-48fb-b115-00d31e5a2f21 service nova] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:21:02.067965 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2feb6655-4b5e-48fb-b115-00d31e5a2f21 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:21:02.078676 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2feb6655-4b5e-48fb-b115-00d31e5a2f21 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:21:02.080542 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2feb6655-4b5e-48fb-b115-00d31e5a2f21 service nova] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:21:02.100063 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1494/2963] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 10:21:02 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:21:08.398242 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=28 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:21:12.087762 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=30 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:21:12.419181 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182c78a2-3798-4fb8-9cb6-7966a9976ed9 service placement] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:21:12.431447 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182c78a2-3798-4fb8-9cb6-7966a9976ed9 service placement] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:21:12.441907 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00b7b41-3810-4137-81ca-a8d62d12268f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f055f7dc51d43bda9b315ca67f11d58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:21:12.442511 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00b7b41-3810-4137-81ca-a8d62d12268f service nova] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:21:12.442790 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00b7b41-3810-4137-81ca-a8d62d12268f service nova] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:21:12.443057 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00b7b41-3810-4137-81ca-a8d62d12268f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:21:12.453157 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00b7b41-3810-4137-81ca-a8d62d12268f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:21:12.454806 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00b7b41-3810-4137-81ca-a8d62d12268f service nova] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:21:12.465510 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1470/2964] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 10:21:12 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:21:16.048730 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2feb6655-4b5e-48fb-b115-00d31e5a2f21 service nova] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:21:16.057866 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2feb6655-4b5e-48fb-b115-00d31e5a2f21 service nova] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:21:16.067075 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551fb081-9f07-4ce6-8323-e79c83e51751 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e5260e798404e8bb12b25ca8ea97052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:21:16.067075 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551fb081-9f07-4ce6-8323-e79c83e51751 service neutron] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:21:16.067075 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551fb081-9f07-4ce6-8323-e79c83e51751 service neutron] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:21:16.067075 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551fb081-9f07-4ce6-8323-e79c83e51751 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:21:16.073286 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551fb081-9f07-4ce6-8323-e79c83e51751 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:21:16.075319 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551fb081-9f07-4ce6-8323-e79c83e51751 service neutron] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:21:16.084767 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1495/2965] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 10:21:16 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:21:18.133254 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00b7b41-3810-4137-81ca-a8d62d12268f service nova] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:21:18.144978 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00b7b41-3810-4137-81ca-a8d62d12268f service nova] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:21:18.150614 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c81864-7849-4f69-ac55-2960f0bc605b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f055f7dc51d43bda9b315ca67f11d58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:21:18.151216 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c81864-7849-4f69-ac55-2960f0bc605b service nova] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:21:18.151493 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c81864-7849-4f69-ac55-2960f0bc605b service nova] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:21:18.152368 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c81864-7849-4f69-ac55-2960f0bc605b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:21:18.161914 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c81864-7849-4f69-ac55-2960f0bc605b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:21:18.163487 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c81864-7849-4f69-ac55-2960f0bc605b service nova] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:21:18.174010 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1471/2966] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 10:21:18 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:21:26.080648 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=6 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:21:28.169534 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=12 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:22:41.010254 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551fb081-9f07-4ce6-8323-e79c83e51751 service neutron] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:22:41.022428 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551fb081-9f07-4ce6-8323-e79c83e51751 service neutron] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:22:41.027445 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5278d7e5-a2f8-449e-8564-501ec6e5a2af service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80cada22039d4f11935412ee578acd05', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:22:41.028060 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5278d7e5-a2f8-449e-8564-501ec6e5a2af service placement] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:22:41.028336 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5278d7e5-a2f8-449e-8564-501ec6e5a2af service placement] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:22:41.028605 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5278d7e5-a2f8-449e-8564-501ec6e5a2af service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:22:41.037292 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5278d7e5-a2f8-449e-8564-501ec6e5a2af service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:22:41.038862 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5278d7e5-a2f8-449e-8564-501ec6e5a2af service placement] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:22:41.048765 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1496/2967] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 10:22:41 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:22:51.044451 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=6 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:23:23.881232 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c81864-7849-4f69-ac55-2960f0bc605b service nova] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:23:23.900593 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c81864-7849-4f69-ac55-2960f0bc605b service nova] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:23:23.904248 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad940ca-163d-4764-bfe0-402da2f7f8cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e5260e798404e8bb12b25ca8ea97052', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:23:23.904845 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad940ca-163d-4764-bfe0-402da2f7f8cd service neutron] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:23:23.905126 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad940ca-163d-4764-bfe0-402da2f7f8cd service neutron] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:23:23.905399 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad940ca-163d-4764-bfe0-402da2f7f8cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:23:23.975191 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad940ca-163d-4764-bfe0-402da2f7f8cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:23:23.976905 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad940ca-163d-4764-bfe0-402da2f7f8cd service neutron] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:23:23.995841 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1472/2968] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 10:23:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:23:27.447209 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5278d7e5-a2f8-449e-8564-501ec6e5a2af service placement] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:23:27.458046 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5278d7e5-a2f8-449e-8564-501ec6e5a2af service placement] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:23:27.463286 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85ee137-6c5c-4028-bdf7-7bc056db8656 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80cada22039d4f11935412ee578acd05', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:23:27.463908 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85ee137-6c5c-4028-bdf7-7bc056db8656 service placement] REQUEST_METHOD: `GET` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:23:27.464183 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85ee137-6c5c-4028-bdf7-7bc056db8656 service placement] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:23:27.464452 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85ee137-6c5c-4028-bdf7-7bc056db8656 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:23:27.466252 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-e85ee137-6c5c-4028-bdf7-7bc056db8656 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:23:27.478061 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85ee137-6c5c-4028-bdf7-7bc056db8656 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:23:27.480133 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85ee137-6c5c-4028-bdf7-7bc056db8656 service placement] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:23:27.490240 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1497/2969] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 10:23:27 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:23:31.154061 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad940ca-163d-4764-bfe0-402da2f7f8cd service neutron] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:23:31.165576 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad940ca-163d-4764-bfe0-402da2f7f8cd service neutron] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:23:31.171693 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9765f1d0-e63c-4535-8bcc-78a9752d26db service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80cada22039d4f11935412ee578acd05', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:23:31.178708 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9765f1d0-e63c-4535-8bcc-78a9752d26db service placement] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:23:31.178708 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9765f1d0-e63c-4535-8bcc-78a9752d26db service placement] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:23:31.178708 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9765f1d0-e63c-4535-8bcc-78a9752d26db service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:23:31.181094 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-9765f1d0-e63c-4535-8bcc-78a9752d26db service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:23:31.194306 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9765f1d0-e63c-4535-8bcc-78a9752d26db service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:23:31.196044 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9765f1d0-e63c-4535-8bcc-78a9752d26db service placement] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:23:31.207178 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1473/2970] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 10:23:31 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:23:31.325276 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85ee137-6c5c-4028-bdf7-7bc056db8656 service placement] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:23:31.339016 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85ee137-6c5c-4028-bdf7-7bc056db8656 service placement] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:23:31.343808 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31360ac3-0a9c-4fcb-959f-4b8e2448462d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:23:31.344460 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31360ac3-0a9c-4fcb-959f-4b8e2448462d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:23:31.344725 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31360ac3-0a9c-4fcb-959f-4b8e2448462d admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:23:31.344999 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31360ac3-0a9c-4fcb-959f-4b8e2448462d admin admin] PATH_INFO: `/v3/users/89351fa3299b41c5b543f0ccdf9f7527` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:23:31.346115 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31360ac3-0a9c-4fcb-959f-4b8e2448462d admin admin] RBAC: Authorizing `identity:delete_user(user_id=89351fa3299b41c5b543f0ccdf9f7527)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:23:31.348078 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31360ac3-0a9c-4fcb-959f-4b8e2448462d admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:23:31.486518 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-31360ac3-0a9c-4fcb-959f-4b8e2448462d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '89351fa3299b41c5b543f0ccdf9f7527'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:23:31.487210 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-31360ac3-0a9c-4fcb-959f-4b8e2448462d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '89351fa3299b41c5b543f0ccdf9f7527'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:23:31.575221 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-31360ac3-0a9c-4fcb-959f-4b8e2448462d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '89351fa3299b41c5b543f0ccdf9f7527'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:23:31.575855 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-31360ac3-0a9c-4fcb-959f-4b8e2448462d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '89351fa3299b41c5b543f0ccdf9f7527'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:23:31.583164 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-31360ac3-0a9c-4fcb-959f-4b8e2448462d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '89351fa3299b41c5b543f0ccdf9f7527'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:23:31.601545 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1498/2971] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 24 10:23:31 2022] DELETE /identity/v3/users/89351fa3299b41c5b543f0ccdf9f7527 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 10:23:31.862492 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9765f1d0-e63c-4535-8bcc-78a9752d26db service placement] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:23:31.874076 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9765f1d0-e63c-4535-8bcc-78a9752d26db service placement] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:23:31.931550 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b881356d-e991-437d-9962-49691544328c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:23:31.932229 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b881356d-e991-437d-9962-49691544328c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:23:31.932512 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b881356d-e991-437d-9962-49691544328c admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:23:31.932787 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b881356d-e991-437d-9962-49691544328c admin admin] PATH_INFO: `/v3/projects/e8f7d973a28745d0a1109ce56927b450` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:23:31.933895 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b881356d-e991-437d-9962-49691544328c admin admin] RBAC: Authorizing `identity:delete_project(project_id=e8f7d973a28745d0a1109ce56927b450)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:23:31.935826 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b881356d-e991-437d-9962-49691544328c admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:23:32.038795 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-b881356d-e991-437d-9962-49691544328c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e8f7d973a28745d0a1109ce56927b450', 'request_id': 'req-b881356d-e991-437d-9962-49691544328c'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:23:32.045425 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-b881356d-e991-437d-9962-49691544328c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e8f7d973a28745d0a1109ce56927b450', 'request_id': 'req-b881356d-e991-437d-9962-49691544328c'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:23:32.047375 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:23:32.051284 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-b881356d-e991-437d-9962-49691544328c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e8f7d973a28745d0a1109ce56927b450', 'request_id': 'req-b881356d-e991-437d-9962-49691544328c'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:23:32.113602 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-b881356d-e991-437d-9962-49691544328c admin admin] The token cache is being invalidate because project e8f7d973a28745d0a1109ce56927b450 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:23:32.113850 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-b881356d-e991-437d-9962-49691544328c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:23:32.114460 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-b881356d-e991-437d-9962-49691544328c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:23:32.115944 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1474/2972] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 24 10:23:31 2022] DELETE /identity/v3/projects/e8f7d973a28745d0a1109ce56927b450 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:23:32.120474 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31360ac3-0a9c-4fcb-959f-4b8e2448462d admin admin] Authenticating user token {{(pid=81602) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:23:32.121914 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-31360ac3-0a9c-4fcb-959f-4b8e2448462d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:23:32.135928 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31360ac3-0a9c-4fcb-959f-4b8e2448462d admin admin] Validating token access rules against request {{(pid=81602) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:23:32.190180 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1979a952-22e7-4391-9d59-2ba6d26b40cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81602) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:23:32.190814 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1979a952-22e7-4391-9d59-2ba6d26b40cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:23:32.191094 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1979a952-22e7-4391-9d59-2ba6d26b40cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:23:32.191468 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1979a952-22e7-4391-9d59-2ba6d26b40cb admin admin] PATH_INFO: `/v3/users/13994c0a59ef4d9ca4956b06a9f07099` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:23:32.192611 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1979a952-22e7-4391-9d59-2ba6d26b40cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=13994c0a59ef4d9ca4956b06a9f07099)` {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:23:32.194511 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1979a952-22e7-4391-9d59-2ba6d26b40cb admin admin] RBAC: Authorization granted {{(pid=81602) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:23:32.395652 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-1979a952-22e7-4391-9d59-2ba6d26b40cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '13994c0a59ef4d9ca4956b06a9f07099'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:23:32.396287 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-1979a952-22e7-4391-9d59-2ba6d26b40cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '13994c0a59ef4d9ca4956b06a9f07099'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:23:32.483664 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-1979a952-22e7-4391-9d59-2ba6d26b40cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '13994c0a59ef4d9ca4956b06a9f07099'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:23:32.484389 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-1979a952-22e7-4391-9d59-2ba6d26b40cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '13994c0a59ef4d9ca4956b06a9f07099'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:23:32.489986 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:23:32.492098 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.notifications [None req-1979a952-22e7-4391-9d59-2ba6d26b40cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '13994c0a59ef4d9ca4956b06a9f07099'} {{(pid=81602) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:23:32.509695 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1499/2973] 10.222.0.16 () {58 vars in 1224 bytes} [Wed Aug 24 10:23:32 2022] DELETE /identity/v3/users/13994c0a59ef4d9ca4956b06a9f07099 => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:23:32.754046 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b881356d-e991-437d-9962-49691544328c admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:23:32.755607 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-b881356d-e991-437d-9962-49691544328c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:23:32.769004 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b881356d-e991-437d-9962-49691544328c admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:23:32.826948 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c0fb55-974f-4802-927e-529ac41b38a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1828412cb04465dbfa15b26582aa6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e359feabfca34b0e8852e124e6b4b5a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:23:32.827599 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c0fb55-974f-4802-927e-529ac41b38a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:23:32.827911 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c0fb55-974f-4802-927e-529ac41b38a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:23:32.828186 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c0fb55-974f-4802-927e-529ac41b38a6 admin admin] PATH_INFO: `/v3/projects/922ea74720354bc488557a97ec9fe087` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:23:32.829300 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c0fb55-974f-4802-927e-529ac41b38a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=922ea74720354bc488557a97ec9fe087)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:23:32.831266 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c0fb55-974f-4802-927e-529ac41b38a6 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:23:33.003515 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-a9c0fb55-974f-4802-927e-529ac41b38a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '922ea74720354bc488557a97ec9fe087', 'request_id': 'req-a9c0fb55-974f-4802-927e-529ac41b38a6'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:23:33.009803 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-a9c0fb55-974f-4802-927e-529ac41b38a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '922ea74720354bc488557a97ec9fe087', 'request_id': 'req-a9c0fb55-974f-4802-927e-529ac41b38a6'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:23:33.015452 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-a9c0fb55-974f-4802-927e-529ac41b38a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '922ea74720354bc488557a97ec9fe087', 'request_id': 'req-a9c0fb55-974f-4802-927e-529ac41b38a6'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:23:33.112327 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-a9c0fb55-974f-4802-927e-529ac41b38a6 admin admin] The token cache is being invalidate because project 922ea74720354bc488557a97ec9fe087 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:23:33.112623 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-a9c0fb55-974f-4802-927e-529ac41b38a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:23:33.113316 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-a9c0fb55-974f-4802-927e-529ac41b38a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:23:33.114941 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1475/2974] 10.222.0.16 () {58 vars in 1233 bytes} [Wed Aug 24 10:23:32 2022] DELETE /identity/v3/projects/922ea74720354bc488557a97ec9fe087 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:23:42.655705 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=8,keystone:DELETE=2 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:23:43.108954 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=60,keystone:DELETE=9,keystone:INSERT=2 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:24:42.000610 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5f42c8-60b2-4c4c-adc1-9d563f4addf2 None None] REQUEST_METHOD: `POST` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:24:42.001231 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5f42c8-60b2-4c4c-adc1-9d563f4addf2 None None] SCRIPT_NAME: `/identity` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:24:42.001231 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5f42c8-60b2-4c4c-adc1-9d563f4addf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81602) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:24:42.030584 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.auth.core [None req-2b5f42c8-60b2-4c4c-adc1-9d563f4addf2 None None] MFA Rules not processed for user `80cada22039d4f11935412ee578acd05`. Rule list: `[]` (Enabled: `True`). {{(pid=81602) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:24:42.031874 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG keystone.common.fernet_utils [None req-2b5f42c8-60b2-4c4c-adc1-9d563f4addf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81602) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:24:42.094235 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: [pid: 81602|app: 0|req: 1500/2975] 10.222.0.17 () {58 vars in 1015 bytes} [Wed Aug 24 10:24:41 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:24:42.107705 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c0fb55-974f-4802-927e-529ac41b38a6 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:24:42.109262 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-a9c0fb55-974f-4802-927e-529ac41b38a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:24:42.124314 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c0fb55-974f-4802-927e-529ac41b38a6 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:24:42.129532 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb6df73-3827-4896-85de-91ee5b9c0553 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80cada22039d4f11935412ee578acd05', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c467e88e2fb0491bacefcf6d2ce326d9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:24:42.130139 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb6df73-3827-4896-85de-91ee5b9c0553 service placement] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:24:42.130424 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb6df73-3827-4896-85de-91ee5b9c0553 service placement] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:24:42.130698 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb6df73-3827-4896-85de-91ee5b9c0553 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:24:42.139892 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb6df73-3827-4896-85de-91ee5b9c0553 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:24:42.141506 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb6df73-3827-4896-85de-91ee5b9c0553 service placement] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:24:42.152526 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1476/2976] 10.222.0.16 () {60 vars in 1365 bytes} [Wed Aug 24 10:24:42 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:24:52.085080 dv-cinder-850321-5-iscsi devstack@keystone.service[81602]: DEBUG dbcounter [-] [81602] Writing DB stats keystone:SELECT=33 {{(pid=81602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:24:52.147561 dv-cinder-850321-5-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=6 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}